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( 1 ) 



Central Missouri State University (Central) proposed to FIPSE to develop and pilot a 
transformational Continuous Process Improvement (CPI) Model, as a replacement for the course-credit- 
completion paradigm, In ten academic departments - expanded to IS departments in 1992. Based on 
the principles of Alvemo’s assessment-as-learning and industries' quality management, the CPI model is 
student learning centered, outcomes-d riven, performance-based and Is Integrated bv continuous 
assessment and feedback . The FIPSE project, Phase IV (1991-94) of a Decade of Transformation, built 
upon several years of exploration, discovery, experimentation (research), faculty education (cultural 
change), policy development and planning. The project was driven by, and evaluated against, ten 
explicit goals which are reported alongside of results in the report. The pilot demonstrated that faculty 
can fundamentally change from the course credit paradigm to a student learning centered, outcomes- 
d riven, performance-based, continuous assessment model. Central’s administration and faculty leaders 
are proceeding to implement Phase V - generalization and implementation of CPI throughout the 
university. Central has applied to FIPSE for a dissemination grant to work with eight adapting institutions 
to develop additional exemplars for systemic reform of universities. A comprehensive program of 
comparative testing of the CPI model at these eight Institutions against the course-credit model at eight 
comparable institutions will follow this phase. 
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A. Project Overview : 

Central Missouri State University (Central) proposed to FIPSE to develop a transformational 
Continuous Process Improvement (CPI) Model and to implement this CPI model in ten academic 
departments in three years, 1991-94. The project was expanded to 15 departments in 1992 (Exhibit 1). 
The CPI model is student learning centered, outcomes-driven. performance-based and is Integrated bv 
continuous assessment and feedback (Exhibit 2). The FIPSE project, Phase IV (1991-94) of a Decade of 
Transformation (Exhibit 3), built upon several years of exploration, discovery, experimentation (research), 
faculty education (cultural change), policy development and planning. The project was driven by, and 
evaluated against, ten explicit goals for each participating academic department which are reported 
alongside of results in Exhibit 4. These results are elaborated in section V, evaluation, of this final report. 

The answer to the question: “Who was served by this project?" is still developing. Initially, the 
faculty of the ten self selected academic departments (13 major programs). Then, students majoring in 
these programs. Next, the faculty in other departments especially those involved in development of 
Central’s new university studies program. Employers and alumni, who volunteered to review and 
validate student learning outcomes developed by departmental faculty for a major program, were next. 
Finally, professional staff and non academic administrators were influenced as the mission was sharply 
focused on the University’s core learning process. The espoused mission became a highly consensual 
mission-in-use. In other words the organizational culture changed to a focus on student learning, 
explicitly driven by performance-based learning outcomes, integrated by systematic and continuous 
assessment and feedback. 

JL Purpose : 

What was the problem? “Reform in higher educatio has just been pruning the branches of a 
dead tree (O’Bannion, 1995).“ ‘It’s the system stupid (Mullin, 1994)1“ A thorough review of the failures 
(anamolies) of higher education that led to the development of the CPI model is presented in pages 1-5 
of our 1991 proposal to FIPSE. This review provides clear evidence of paradigm failure but no examples 
of paradigm change in higher education. One lesson learned is that the first step in creating a quality 
system is to consciously and explicitly examine the logic of the assumptions underlying the existing 
system. This conviction, that real reform requires breakthrough thinking, has been strengthened by our 
experience. 

The purpose of this FIPSE project was to develop and pilot test a model for postsecondary 
curricular and pedagogical reform; and, to disseminate the model and results to aapropriate forums as a 
guide to reform in higher education. The model was conceived as an alternative paradigm to replace 
the 100 year old course-credit-completion paradigm. The pilot was to test the feasibility of 
implementation to determine if university faculty would accept such radical and labor-intensive change 
and provide leadership for the next phase of system-wide reform of a complete university. Transforming 
one institution requires transcendent leadership, tireless effort and patience. If that was our only 
purpose, however, the project would not have been justified. Our tong term purpose was and is to be a 
major influence in changing the prevailing paradigm and thus university systems throughout the United 
States. We believe only systemic change is durable. Setting up the next phase (Phase V), which 



includes leadership to develop a set of exemplar institutions, was also an important element of our 
purpose. 

C. Background and Origins : 

What is now known as Central’s CPI model began as an assessment initiative by President Elliott 
in 1985. After two years of definition and trial of traditional assessment programs, the faculty senate 
assessment committee began active involvement in goal setting, planning and resource allocation. 

Three important phases (1987-90) preceeded drafting of the preliminary proposal to FIPSE in October 
1990. What happened in these preliminary phases are outlined in Decade of Transformation (Exhibit 3) 
and described in the body of the report, section C and exhibits 5 and 6. 

After several years of exploration, experimentation and discovery, we became convinced that the 
"system always wins." In other words, unless real systems change occurs, reform efforts in higher 
education will lead to marginal, nondurable improvement. As the Faculty Senate Assessment Committee 
continued to probe the purposes, goals, available models and instruments for assessment, we found few 
convincing answers or role models. Too often people and institutions were using assessment 
instruments without questioning what they were trying to measure and why? Our search for universities 
who effectively used assessment evidence to improve the student learning process was discouraging. 
Most failed to understand the importance of, or actually make the connection between, assessment and 
improvement of student learning and process. This paucity of role models or benchmarks oddly opened 
our eyes and stiffened our resolve to find a comprehensive model that measured the right things and 
systematically improved student learning. We would not be pressured into doing something that didn’t 
make sense. We would not pretend to see new clothes on a naked emperor just because others did. 

Di Project Description 



The CPI model is based on integration of the principles of Alverno's assessment-as-learning and 
industries’ quality management. The CPI model is student learning centered, outcomes-driven. 
performance-based and is Integrated bv continuous assessment and feedback (Exhibit 2). This project, 
described as Phase IV of a five phase Decade of Transformation, 1988-1998 (Exhibit 3), was designed to 
demonstrate fundamental systemic reform in a pilot postsecondary educational institution. Phase IV, the 
FIPSE project (1991-94), thus built on several years of exploration, discovery, experimentation 
(research), faculty education (cultural change), policy development and planning. 

E. Evaluation/Project Results 

The CPI project was very successful at increasing both faculty and student awareness and 
understanding of the CPI model. Over the last four years approximately 70% of Central’s faculty have 
received some type of training in CPI. A significant proportion (10-15%) have received extensive training 
in assessment-as-learning and/or total quality management. Through the use of regular university, 
college, and department level workshops and retreats, faculty are now equipped with new pedagogical 
and assessment techniques. The academic calendar now includes 6 days each year for planning and 
development. Approximately 40% of Central's faculty have added performance-based assessments to 
their classes since the beginning of the CPI project. Students are engaged in a much greater variety of 
learning/assessment exercises in their major courses than previously. Over 5,400 students have been 
exposed to CPI through special workshops, speeches, articles, and in class discussions. All incoming 
freshmen and new faculty receive a formal introduction to CPI during fall orientation. 

Phase V (Exhibit 3) outlines Central’s plans for continuation of the CPI project which basically 
calls for the institutionalization of CPI. Major goals include restructuring of the university, revision of 
budget/resource allocation process, adoption and utilization of CPI principles in remaining major 
programs and university studies, and development of a seamless PK-16 system. 
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Central’s conspicuous successes have been In 1) raising the awareness and understanding of 
students and faculty about the nature of learning and teaching, 2) changing the university culture, 3) 
raising the expectations of faculty and students, and 4) demonstrating that a paradigm shift is possible in 
higher education. 

Central has been less successful at producing documentation of changes in student learning. 

The focus of this project was transformation. Our long term goal Is to produce externally validated, 
performance-based evidence of improved student learning for its graduates. It will take approximately 
three more years until we have a sufficient data base to allow us to demonstrate improvement in student 
performance. 

F. Summary and Conclusions 

In summary, the strategic intervention to penetrate the Central Missouri State University system 
succeeded and we are proceeding to implement Phase V (Exhibit 3) - generalization and 
implementation of CPI throughout the entire university system. At the same time we have applied to 
FIPSE for a dissemination grant to work with eight adapting institutions to develop additional exemplars 
for systemic reform of universities. This next phase (V) will make possible comparative testing of the CPI 
model against the course-credit model. 
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Title: A Continuous Process Improvement (CPI) Model for 

Postsecondary Curricular and Pedagogical Reform Based 
on Assessment-As-Learning 

Directors: Ralph F. Mullin, Director 
D 401 E 816/543-4818 
Central Missouri State University 
Warrensburg, MO 64093 

A. Project Overview. 

Central Missouri State University (Central) proposed to FIPSE to develop a 
transformational Continuous Process Improvement (CPI) Model and to implement this CPI 
model in ten academic departments in three years, 1991-94. The project was expanded to 
1 5 departments in 1 992 (Exhibit 1 ). The CPI model is student learning centered, 
outcomes-driven, performance-based and is integrated by continuous assessment and 
feedback (Exhibit 2). The FIPSE project. Phase IV (1991-94) of a Decade of 
Transformation (Exhibit 3), built upon several years of exploration, discovery, 
experimentation (research), faculty education (cultural change), policy development and 
planning. The project was driven by, and evaluated against, ten explicit goals for each 
participating academic department which are reported alongside of results in Exhibit 4. 
These results are elaborated in section E, evaluation, of this final report. 

The answer to the question: "Who was served by this project?" is still developing. 
Initially, the faculty of the ten self selected academic departments (13 major programs). 
Then, students majoring in these programs. Next, the faculty in other departments 
especially those involved in development of Central's new university studies program. 
Employers and alumni, who volunteered to review and validate student learning outcomes 
developed by departmental faculty for a major program, were next. Finally, professional 
staff and non academic administrators were influenced as the mission was sharply focused 
on the University's core learning process. The espoused mission became a highly 
consensual mission-in-use. In other words the organizational culture changed to a focus 
on student learning, explicitly driven by performance-based learning outcomes, integrated 
by systematic and continuous assessment and feedback. 

B. Purpose/ Problem . 

1. Problem. What was the problem? "Reform in higher education has just been pruning 
the branches of a dead tree (O'Bannion, 1995)." "It's the system stupid (Mullin, 1994)1" 

A thorough review of the failures (anamolies) of higher education that led to the 
development of the CPI model is presented in pages 1-5 of our 1991 proposal to FIPSE. 
This review provides clear evidence of paradigm failure but no examples of paradigm 
change in higher education. One lesson learned is that the first step in creating a quality 
system is to consciously and explicitly examine the logic of the assumptions underlying the 
existing system. This conviction, that real reform requires breakthrough thinking, has been 
strengthened by our experience. Our understanding of the problem has not fundamentally 
changed but has elaborated and deepened. 

2. Purpose. The purpose of this FIPSE project was to develop and pilot a model for 
postsecondary curricular and pedagogical reform; and, to disseminate the model and 
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results to appropriate forums as a guide to reform in higher education. The model was 
conceived as an alternative paradigm to replace the 100 year old course-credit-completion 
paradigm. The pilot was to test the feasibility of implementation to determine if university 
faculty would accept such radical and labor-intensive change and provide leadership for the 
next phase of system-wide reform of a complete university. Transforming one institution 
requires transcendent leadership, tireless effort and patience. If that was our only 
purpose, however, the project would not have been justified. Our long term purpose was 
and is to be a major influence in changing the prevailing paradigm and thus university 
systems throughout the United States. We believe only systemic change is durable. This 
conviction has been strengthened by our experience. Setting up the next phase (Phase V), 
which includes leadership to develop a set of exemplar institutions, was also an important 
element of our purpose. 

3. Potential for adapting In preparing the FIPSE Dissemination Grant Proposal, now 
under consideration, we discovered a high level of excitement for systemic change from 
the eight adapting institutions recruited. Several were quite familar with Central's CPI 
model. The balance quickly saw the validity of the model and applicability to their need for 
quantum improvement in student learning. Before the project has even reviewed these 
schools are proceeding with plans. John Bigelow at Boise State has cleared his release 
time and has secured commitments of key chairs to proceed. Andrea Leskes, 

Northeastern University, has developed a presentation on the dissemination project 
(inviting us to participate, at the National Council of Colleges of Arts and Sciences in 
November. Three schools, Babson, DePaul and Samford, are participating with Central at 
an AACSB Continuous improvement Symposium in Philadelphia in October. There is 
genuine leadership for systemic reform here. The CPI model and Central's experience 
appears to be an effective catalyst. 

4. Lessons Learned 



a. Paradiam/Core Process . Reform programs must challenge the fundamental 
structure of higher education. The course credit completion model is more a 
device for making teaching easier than for improving learning. Universities have 
failed to be effective learning organizations and much of that failure is attributable 
to the course-credit model. Simply overlaying TQM principles over the flawed 
course-credit structure will fail to produce lasting change in organizational culture 
and behavior. A theoretically-sound model of the core learning process is a 
necessary condition for reform. 

b. Principles . The assumptions of AAL and principles of TQM provide invaluable 
guidance; they only instruct practice, however, and must meet what we call the 
three test-interpretation, integration, and implementation. For example, "focus on 
customer" must be interpreted and understood in a comprehensive way as 
meaning anyone whom you serve, who uses or benefits from the value of your 
effort. If one narrowly interprets this principle to simply identify the student or the 
employer as the customer, the principle has lost its power to explain and guide. 

We found failures were invariably a result of not attending to all relevant principles 
simultaneously (integration). TQM is not simplistic; it is complex and requires 
systematic application. Quality results are still dependent on quality 
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implementation. TQM is not a magic formula. Careful focus on implementing the 
details is still necessary. 

c. Leadership . It takes both, top-down and bottom-up leadership. In higher 
education, faculty must lead-faculty who value learning, teaching, and service 
enough to want to improve. Right motives are essential when asking people to 
transcend their self interests and do the hard work of reform. Faculty leaders will 
be most respected by other faculty if they are viewed as independent and internally 
motivated. Top-down leadership, however, must empower faculty to lead, 
encourage innovation, and provide funding and freedom to experiment. 

d. Focus . We learned to fund hot spots of energy, concentrating time and money on 
faculty teams that were producing results. Energy was not wasted on faculty who 
refused to be open to logical argument and change. 

e. Patience . Torbert (1 992) suggests one reason TQM fails is because it requires 
"multiple transformation phases over a decade of time." Planning must be done in 
terms of decades. Reform leaders must emphasize there is no quick fix. Cultural 
change, involving everyone in an institution, will demand patience and transcending 
leadership. 

f. Money . Seed money, in our case from FIPSE and Central, may be necessary to 
overcome inertia, get new processes going, and flexibly fund hot spots of energy. 
External funds also strengthen credibility. 

g. Training . The need for training is constant and eternal. At each progressive stage, 
more and more skills and knowledge are needed and sought by faculty. 

h. Faculty Involvement . We learned faculty will make quality improvement succeed 
or fail. Faculty make design and implementation decisions. The CPI project is 
faculty owned. Project directors are only coordinators and facilitators. Decisions 
made collaboratively by faculty in departments result in intrinsic motivation for 
change, improvement, and renewal. Faculty must leadl 



C. Background and Origin of Model . 

1 . Background: Central Missouri State University is a comprehensive university 

comprised of four colleges, 34 departments, and 1 1 ,400 students. Central is located in 
Warrensburg, a small town of 15,000 only 50 miles southeast of Kansas City. Central has 
grown from a State Normal School through three name changes. It is a leader in 
discipline-specific accreditations within the Missouri system. Central offers a wide variety 
of undergraduate, graduate and education specialist degree programs. 

When Ed Elliott began his tenure as president in 1985, the organizational culture might 
have been described as low in trust and openness, suspicious of administration, individual 
competitive, perception of the reward system as political. The result was that faculty 
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avoided risk and high performance behaviors including reluctance of faculty to assume 
responsibility. 

In 1985-86, Central, in response to calls by state government for greater institutional 
accountability established the following: (1) assessment of math and writing skills of 
entering freshmen for placement; (2) a writing-across-the-curriculum program; (3) 
assessment of writing in the English department; (4) further development of Central's 
Educational Development Center. During the same time period. Central's Faculty Senate 
also established an assessment committee. As this committee worked on developing an 
approach to assessment, members became increasingly frustrated by: (1) requests for a 
test to assess a major program for which faculty could not clearly define the outcomes to 
be assessed, (2) the lack of congruence between the content of standardized tests and 
content being taught (content and construct validity), and (3) the lack of usefulness of 
standardized test results either to improve student learning or to evaluate programs. Over 
and over again members raised the same questions: "What is it that we should be 
measuring? Why? And to what end?" This increasing frustration served to clarify the 
needs for: (1) an explicit definition of the purpose and ends of assessment, (2) explicit 
outcomes of student learning, and (3) a comprehensive assessment model to explain the 
complex learning and assessment process. In the fall of 1987, the committee began an 
intentional search for a model and funded the first phase of a program of experimentation. 
(Exhibits 5 and 6). 

2. Model Discovery and Development, June 1988-June 1991 

This three-year phase was characterized by: (1) discovery of a sound model, (2) 
faculty exposure, understanding, and acceptance, (3) experimentation with alternative 
assessment methods, (4) discovery that the principles of total quality management (TQM) 
and assessment as learning (AAL) are highly congruent, (5) recognition that a fundamental 
cultural change was evolving which would require multiple transformation phases, and (6) 
design and development of goals, strategies, and program. 

The search for a sound model to guide reform quickened in June 1 988 at the AAHE 
Third Annual Assessment Conference upon discovering Alverno College's assessment-as- 
learning approach. Focusing on student learning resolved purpose or mission. 
Understanding assessment as an integral element in the learning process itself, coupled 
with Alverno's logically sound set of assumptions (Exhibit 26), provided a clear conceptual 
basis for a model. 

Although a few faculty received the model positively, most did not grasp the profound 
implications of the assumptions and the comprehensiveness of the model. Recognizing 
that changing faculty's basic assumptions about learning would be difficult and time- 
consuming, the assessment committee made faculty exposure to assessment experiences 
a priority. Over 50 faculty participated in a variety of assessment conferences in the next 
two years. By fall, 1 989, the committee framed a set of principles defining assessment at 
Central, and sponsored two important on-campus conferences: a series of seven meetings 
conducted by Peter Ewell, involving over 200 faculty; and, a three-day assessment-as- 
learning workshop conducted by a team of four faculty from Alverno with 1 25 faculty 
participating. Sixty-seven selected faculty, representing four colleges and seventeen 
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disciplines, have received intensive training at week-long Alverno workshops. Forty-seven 
of these were prior to the beginning of the FIPSE project -- four in May, 1989, seventeen 
in June, 1990, and twenty-six in June, 1991 (Exhibit 5). 

An experimentation program was planned using two sets of measures developed by 
the American Assembly of Collegiate Schools of Business's (AACSB) Outcomes 
Measurement Project (1987). The purpose of the experimentation program was to learn 
more about testing versus performance-based assessment, and the usefulness of each for 
both student learning and program evaluation. The results of the traditional test over a 
common body of knowledge yielded no significant difference between means from 
comparable schools; thus, no data useful for either program evaluation or student learning 
resulted. The performance-based assessments were used in three phases: (1) a one group 
pre-/post- design with 35 students, fall semester 1988, (2) a Solomon four design test 
with 276 students, spring 1990, and (3) an application of assessment methods in fifteen 
different courses in four colleges, 1990-91 school year. As an example of how the faculty 
educational effort interacted with the experimental program, faculty who taught the fifteen 
courses in Phase III were the evaluation team for the Phase II research. The evaluation 
team committed to attend the five-day Alverno workshop, teach a course using AAL 
methods (Phase III), and later mentor other faculty in Phase IV (CPI/FIPSE project). The 
empirical results of the two field experiments, reported in Managerial Skills (Mullin, Shaffer 
& Grelle, 1991), were highly supportive of the general hypothesis that students can 
develop complex skills in an academic setting while learning content knowledge. These 
results conveyed that our approach to assessment and reform was scholarly and rigorous. 

From 1986 to the present leaders in assessment were brought to campus in an 
attempt to raise faculty and staff understanding of assessment, pilot assessment-as- 
learning projects were conducted and faculty interest in assessment grew. In 1990, the 
discovery of the striking conceptual similarities between TQM (Exhibit 2c) and assessment 
as learning (Exhibit 2b) enriched our thinking about a model and how to effectively 
communicate with employers. By early fall, 1990, we outlined a strategy to design and 
implement a model of AAL and TQM at Central. From the outset, the CPI model was 
intended as a potential replacement for the dominant Course Credit Model. The project 
began in the fall of 1991 with 10 departments committed to restructuring both the 
content of their majors and the means of instruction, based on the principles of the CPI 
model. 

The FIPSE project team represented all four of Central's colleges (seven faculty, two 
assistant deans) and three professional staff departments. All had been heavily involved in 
Phases I through III. Each member has a clearly defined area of responsibility. These ten 
provided the support structure, service, and coordination for the ten departments where 
faculty developed the ten components of the project. 

D. Project Description . 

1 . The CPI Conceptual Framework: 

Central's CPI model builds on the ten assessment-as-learning (AAL) assumptions 
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developed by Alverno College (Exhibit 2b). TQM principles (Exhibit 2c), which are 
highly complementary with AAL, provide additional underpinning for the CPI model 
and provide functional terminology for communicating the CPI model to employers. 
This section of the paper describes the major elements that comprise the CPI model: 
the organizational mission, the core process criterion, and the core process design. 

The CPI model is then described in terms of changes in the framework of higher 
education. 

Organizational Mission 

The TQM literature emphasizes that organizational change begins with re-envisioning 
the mission. The mission must clearly define a simple, focused purpose of the 
organization's existence. This purpose in turn will provide the basis for the design of 
the core process. The process of collaboratively re-envisioning and consensually 
agreeing on the organizational mission is as important as the mission statement itself. 
As the TQM literature suggests, such an organizational mission will provide direction 
and motivation only if it is individually and collectively valued. The mission sets the 
stage for re-thinking strategies, while the process of collaboration provides a means 
for participants to own the implicit set of values the mission represents. The first 
assumption of AAL embraced in Central's CPI model is that student learning is the 
university's mission, its primary purpose. Successful implementation of CPI depends 
on faculty, staff, and administration consensually valuing this singular mission focused 
on student learning. 

Core Process Criterion 

The university's core process is the system by which the mission, student learning, is 
accomplished. The utility of the core process lies in its ability to produce outcomes 
consistent with the mission. In the CPI model, the faculty of each major program 
specify the criteria for the core process in terms of a comprehensive set of 
performance-based student abilities, or "outcomes". The outcomes involve complex 
abilities which integrate knowledge, skills, and attitudes (KSAs). These outcomes 
define the desired "ends" of the core process, focus all learning activities and 
methods, and are the criteria for measuring the effectiveness of the core process 
(Exhibit 2b, Assumption #2). Faculty take ownership through the consensual process 
of defining both general and specific outcomes. Articulating the outcomes and making 
them public allows other important constituents (employers, alumni, graduate schools, 
etc.) to participate in outcomes validation and to share ownership (Exhibit 2b, 
Assumption #9). 

Core Process Design 

The CPI design for classroom learning, established on principles of AAL, involves 
faculty and students. Faculty: (1) define performance-based student abilities to be 
learned and demonstrated in the course, (2) write criteria which specify the expected 
characteristics and quality of performance for each ability, (3) observe student 
performance based on the stated abilities and specific criteria, (4) assess student 
performance based on expert judgment, and (5) provide developmental feedback to 
the student against the specific criteria. Students: (6) process the feedback 
cognitively, and (7) repeat the performance. The CPI design of the core process is 
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depicted in Exhibits 2a and 2d). This learning process is supported by most cognitive 
and behavioral learning theory, (e.g., Bandura, 1977; Luthans & Kreitner, 1985). 

In the CPI model, students acquire complex abilities developmentally when they "make 
an action out of knowledge" based on specific performance criteria, receive feedback 
based on expert judgment, and cognitively adjust to improve their subsequent 
performance (Exhibit 1, Assumption #6). Students learn from repeated opportunities 
to practice desired abilities in various situations and contexts. These multiple 
experiences facilitate the retention of knowledge and the transfer of abilities to new 
situations at higher levels of generalization (Exhibit 2b, Assumption #5). 

Observation and assessment of student performances is not only a series of discrete 
measurements, but a continuous and integral part of the students' learning process 
design (Exhibit 2b, Assumption #4). Assessment produces feedback as a part of a 
continuous improvement process for each individual student. Because general 
outcomes involve multidimensional, complex abilities, students must perform and be 
assessed in a variety of modes and contexts. (Exhibit 2b, Assumption #5) 

In the CPI model, courses and sets of courses (curriculum) are the means that provide 
a structure for student learning. Each course is developed as an element in a coherent 
curriculum, structured so that students acquire complex abilities developmentally 
across courses. Although feedback from student performances guides student 
learning, it also provides information for continuous review and improvement of 
courses and curriculum (Exhibit 2b, Assumption #9). Creation, maintenance, and 
improvement of a coherent curriculum is iterative, continuous, and dynamic (Exhibit 
2b, Assumption #8). Management of the curricular process depends on faculty 
investment in a community of learning and judgment (Exhibit 2b, Assumption it 7 ). 

CPI Paradigm Shift 

Degree requirements drive student and faculty action in the learning process. In the 
traditional course-credit model, degree requirements are stated exclusively in terms of 
required courses. The set of required courses serves as the criterion for the core 
learning process, as the organizing principle for structuring the major program, and as 
the end of the educational process. Since the course credit criterion provides no 
systematic mechanism to coordinate and integrate student learning developmentally 
across the curriculum, courses typically evolve as discrete packets of content 
knowledge. Neither faculty nor students have the means or the impetus to transfer or 
generalize knowledge and skills from one course to another. 

The CPI model shifts the measure of student competency and the organizing principle 
for the degree program from the completion of courses to the demonstration of 
performance-based abilities. In the CPI model, the required curriculum becomes a 
means for systematic, developmental student learning while performance-based 
student abilities are its ends. Changing the role of curriculum from ends to means is 
subtle but profound. It is a paradigm shift which compels at least eight fundamental, 
systemic changes in program design and pedagogy. 

First, the CPI description of the major in terms of ability-based performances provides 
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a common organizational principle for faculty and student action. Individual courses 
are unified around performance-based abilities that describe the major program, such 
that complex communication, interpersonal, thinking, valuing, and technical skills are 
learned developmentally across a variety of context and content specific courses. 

Second, the CPI public articulation of the performance-based abilities allows 
involvement of all "customers and suppliers" in designing the learning process. All 
interested constituencies, including students, alumni, employers, and other faculty can 
review and assess outcomes for relevancy and importance. The outcomes also 
provide the means for developing relationships with elementary and secondary 
educators (suppliers). 

Third, the CPI emphasis for student learning shifts from the recollection of content 
knowledge to student demonstration of an integrated set of knowledge, skills, and 
attitudes (KSAs). This shift in emphasis enhances the value of "content knowledge", 
by promoting learning at the higher level of understanding. The preeminence of 
student performance is consistent with the AAL Assumption #2, "Education goes 
beyond knowing to being able to do what one knows" (Exhibit 2b). 

Fourth, the CPI focus on student abilities forces systematic consideration of the 
developmental nature of the learning process. Curriculum, as the "means", must be 
designed to foster student progress from basic to more advanced proficiency levels, 
and to engender student learning of complex abilities as a developmental process. By 
understanding the performance-based abilities expected of them, students are better 
able to recognize how skills and knowledge are to be transferred and developed across 
courses. The public statement of these abilities provides an explicit road map to guide 
individual student learning, thereby increasing both the opportunity and motivation for 
students to learn. 

Fifth, CPI, allows co-curricular learning activities to directly complement formal course 
work. Leadership of student organizations, internships, developmental seminars, work 
experience, and other campus activities provide valuable learning opportunities which 
contribute to the development of desired student abilities. Instructors may help 
students to identify appropriate co-curricular learning opportunities related to student 
performance expectations and individual student abilities. 

Sixth, CPI makes assessment an integral, continuous part of the learning process 
instead of inspection and a grade at the end. Assessment is a continuous process, 
not a series of discrete steps. Expert judgment based on the observation of student 
performances provides feedback for performance improvement. Students learn to 
regard assessment of performances in terms of constructive opportunity for 
improvement rather than as a mechanism to determine a grade. 

Seventh, and most importantly, CPI encourages change in classroom teaching 
methods. Instead of learning exclusively "about" the content knowledge related to a 
discrete course, students must learn to apply, assess, and reflect on content 
knowledge in a variety of contexts. 
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Eighth, Although CPI feedback is primarily used for individual student development, 
faculty also use feedback. Over time, the observation of student performances 
provides rich information about the relationship between student performances 
instruction, and curriculum to continuously improve the core process. 

These eight fundamental differences are summarized in Exhibit 4, which presents a 
comparison between CPI and the course credit model. These differences require major 
changes in faculty values and behavior, and the organizational culture. 

2. Implementing the CPI Model 

In January, 1991, FIPSE invited a final proposal, requiring us to develop details on the 
design, goals and program. Decisions could not be made on many of the goals 
without actually gaining commitments from all faculty in participating departments. A 
draft proposal was sent to all 34 departments inviting them to consider participation. 
Preliminary meetings with department chairs were followed by meetings with the 
entire faculty of departments showing interest. Faculty were asked to read the 
preliminary proposal in advance to better understand specific commitments. Ten 
departments (twelve academic majors) with balanced representation from the four 
colleges (three majors from each college) volunteered. As faculty experienced 
success, the number of departments involved was to increase. This broad 
involvement of faculty, an essential TQM principle, was a key to success. 

From March to September, 1991, planning proceeded on the assumption the FIPSE 
grant would be approved. Before FIPSE approval in September, much of Central's 
matching money ($50,000) had been committed for training and release time for the 
1 2 departmental project coordinators. The project was launched. The 14 
departments, 19 programs, and 200 faculty, including second year additions (1992- 
93), are shown in exhibit 1. The goals of this three-year phase (1991-94) are shown 
in Exhibit 4. 

New initiatives in year two (1992-93) included expanding the project by: (1) adding 
eight new programs, (2) decentralizing by greater involvement of the college deans 
who now meet with their CPI chairs and departmental project coordinators monthly, 

(3) initiating CPI/TQM training and planning with administrative departments, (4) 
developing a preferred supplier program with high school principals and teachers, (5) 
faculty participation in industry TQM training programs, and (6) further development of 
customer relations guided by a Quality Advisory Team from TQM companies. 



E. Project Results/Evaluation . 

Central's original FIPSE proposal listed ten goals the project was to address. Since CPI is a 
systems reform model, these ten goals focused on changes in the system. Thus focus on 
faculty learning and change. Although Central's long term goal is to produce externally 
validated, performance-based evidence of improved student learning for its graduates, the 
FIPSE project goals centered on the necessary prerequisite for such improvement, reform 
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of the system. As Motorola's Bob Galvin (1993) asserts, "First one must have a quality 
system in piace. Then one must measure the right things." Our results accordingly are 
primarily "process" results. This section documents the process changes at Centra! in the 
ten goal areas described previously. 

The evaluation of the project was conducted by Central's Coordinator of Assessment and 
Testing Services. The Assessment Coordinator reports directly to the Provost and worked 
closely with him during both the formative and summative stages of the evaluation. A 

one-page summary of results to date on the ten components for Phase IV (1991-94) is 
presented in Exhibit 4b. 

GOAL 1 . Model. 

Results . In 1991, Central committed to the Fund for the Improvement of Postsecondary 
Education (FIPSE) to design, introduce, and implement the CPI model in "ten academic 
departments and fourteen major programs on four colleges." In Three and one-half years, 
nineteen departments and twenty five programs are directly involved in the process of 
implementing the model (Exhibit 1). Others, not formally involved in the FIPSE project, are 
now in full gear, e.g.. Agriculture, Marketing and Legal Studies, Graphics. These 
departments represent over 45% of the faculty at Central Missouri State University. 

In addition to the implementation of the CPI model in the aforementioned departments. 
Central has expanded the model into its general education (university studies) program as 
well (See Exhibit 7) for a description of a proposal recently submitted to our Coordinating 
Board for Higher Education to expand CPI into university studies.). This extension of the 
model was not part of the original ten goals but was a natural byproduct of the CPI 
implementation process. As more and more departments became involved in CPI, the 
need to extend the model to include university studies was recognized by the faculty. Our 
revised university studies program establishes exit performance-based competency levels 
in the four general outcomes of thinking, communicating, valuing, and interacting for all 
students. All faculty scheduled to teach in the revised program, which begins the summer 
of 1996, must teach to and assess at least two of these four outcomes using CPI 
principles. 

Results from a survey distributed to faculty in CPI/FIPSE departments/majors suggest the 
cultural intervention is working in terms of attitudinal and behavioral change (Exhibit 8a). 
The questionnaire consisted of 34 statements. An examination of the responses on the 
first 1 5 items indicated a high degree of understanding and support for the CPI project by 
faculty. On a scale of 1-5 (Strongly Disagree - Strongly Agree), the average faculty 
response to items 1-10, 12, and 15 was over 4.10, indicating consistent agreement, 
understanding, and valuing of the CPI principles by faculty. 

Items 1 6-34 provide a measure of the degree of actual involvement in the new 
teaching/learning practices by the faculty from both the individual and group (department) 
perspective. The strongest point of agreement was for item 1 6, which measured the 
degree to which faculty/departments had collaboratively developed outcomes for their 
programs (mean of 4.1 1 and 4.19 respectively). This was as expected since define 
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outcomes as a first step. The differences between the responses by the faculty given 
from the perspectives of self and department were negligible, suggesting that the faculty 
perceived their participation in the project to be equal to that of the other members of their 
department. These results provide evidence of the collaborative nature of the outcomes 
development/validation process. 

Another significant CPI offshoot has been the application and use of the CPI model by 
various important committees, task forces, and administrative units. For example, the 
Strategic Planning Council, which is charged with establishing the institution's long term 
goals and strategies, has adopted several long term objectives that are a direct outgrowth 
of the CPI model. For example, the Council is recommending a change in the reward 
system that would allocate additional resources to departments which can provide 
evidence of improved student learning towards either departmental or university studies 
outcomes. The departmental Annual Reports have been revised and now require each 
academic department to report progress on the development and implementation of 
departmental outcomes, assessment methods, pedagogy, curriculum matrices, and faculty 
development. The Student Information Systems task force will use CPI principles to 
restructure the student data base. The use of CPI to rework the student data base will 
result in a major shift in what the university as an institution considers to be important 
student information (Exhibit 9). A Quality Self Assessment team has begun the process of 
performing a rigorous self assessment of the entire institution using the seven Malcome 
Baldridge National Quality Award criteria. Official university commercials and publications 
used to attract students, refer to CPI as Central's model of learning and teaching (Exhibit 
10 ). 

Evaluation. These and other results indicate that Central has made tremendous progress 
towards institutionalization of CPI (a major purpose of Phase V). We have clearly 
accomplished our first purpose - developing and implementing the CPI model at Central. 

Another purpose was to show that a comprehensive university can initiate and maintain a 
systemic reform effort. The evidence to date would indicate that fundamental changes in 
both the structure and processes of a university are possible. Central has demonstrated 
the commitment and capacity to transform the learning process and restructure the 
university. 

GOAL 2. Outcomes. 

Results. Faculty in nineteen departments have collaborated to produce a set of student 
outcomes for their majors and validated these outcomes with a representative set of 
customers, e.g., graduates, students, employers, experts. An additional three to five 
departments are expected to develop, validate, and publish program specific outcomes for 
their majors in the 1995-96 academic year. 

As previously mentioned, the Faculty Senate University Studies Committee has established 
a set of general outcomes for all our undergraduates. These outcomes will be taught and 
assessed using CPI principles. 

The Financial Aid Office and several departments in the Student Affairs division have 
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established outcomes for their units through use of customer needs analysis. 

Evaluation. Although not all of Central's 34 departments have established outcomes for 
their programs, much progress has been made in improving the relevance, cohesion, and 
direction of a significant number of major programs in a three-year period. By the end of 
next year, over two thirds of Central's departments will have developed and validated 
outcomes for their programs. In addition. Central has developed general education 
outcomes that will be taught and assessed using CPI principles. 

GOAL 3. Assessment. 

Results. Both students and faculty report a striking increase in the number of "products" 
or demonstrations of active learning, e.g., oral-visual presentations, writing assignments, 
portfolios, team projects. A second questionnaire was completed by 460 students at 
Central (Exhibit 8b). This instrument attempted to measure the student's personal 
agreement/disagreement with statements that indicated the learning/assessment activity of 
the faculty in their majors. Basically this survey provided a direct measure of the 
teaching/assessment practices of the faculty who taught the courses in the major. The 
results indicated that the students engaged in a variety of learning/assessment exercises in 
their major courses. The areas of highest involvement were in the use of assessable 
products (e.g., projects, presentations, and portfolios), oral and written assignments, and 
exercises that used higher order thinking skills. The students also reported that their 
instructors had developed explicit outcomes for their courses. These data suggest that the 
perceptions and experiences of the students, both in and out of the classroom, were 
positively affected by CPI. 

This coming academic year, a faculty team will complete and pilot test a comprehensive 
entry (Freshman), performance-based assessment of the four general outcomes which 
drive the new univ3\ersity studies program. The comprehensive assessment is scheduled 
to be used in the fall of 1996 to establish baseline levels for incoming students and to 
identify their strengths and weaknesses for placement purposes. Within the next three 
years. Central will develop comprehensive midpoint and exit assessments of the four 
university studies general outcomes as well. 

This Fall of 1995 Central will implement a planned placement program designed to match 
courses with a student's particular knowledge and general skills level in order to improve 
student learning. 

A new requirement, not one of the original ten FIPSE goals, is that each department will 
utilize a departmental advisory council or board as one means of focusing on customer 
assessment and feedback. This is a partial operationalization of quality function 
deployment (QFD) an important TQM idea. Most of the CPI departments have established 
advisory councils, or are in the process of doing so. Council members, the voices of the 
customers, have been actively involved in comprehensive assessment of students in their 
major. Several of these assessment experiences have stimulated employers, students, and 
faculty in ways not thought possible before CPI. 

Evaluation. We have not met our goal as originally stated. We had hoped that all 
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departments formally involved in the CPI project would have developed and implemented 
comprehensive assessments for their programs by this time. The development of 
comprehensive assessments remains a priority and we expect that ail of the nineteen 
departments currently participating in CPI will have completed midpoint and exit 
assessments by the end of the 1 996-97 school year. 

GOAL 4. Curriculum. 

Results. Fourteen programs have developed and evaluated a curriculum matrix which 
describes when and where course outcomes (knowledge, skills, and attitudes) are to be 
taught and assessed in the major. These matrices reflect changes in course content, both 
knowledge and skills, brought about by the outcomes identification/validation process. 
Twelve of these departments have developed an orientation course for students entering 
their programs that prepares the student for this new learning paradigm. Several of these 
departments have also developed publications which are distributed to students as they 
enter the major which describe the program's goals, philosophy, and assessments. 

The Faculty Senate University Studies Committee hopes to have a curriculum matrix for 
the university studies program completed by the fall of 1 996. 

This year, all new faculty and entering first time, full time freshmen will receive an 
orientation to the CPI model during fall orientation. These sessions will be led by Central's 
Coordinator of Assessment and Testing Services. 

Evaluation. Considerable progress has been made in creating program curricula that are 
integrated, developmental^ sequenced, assessable, and public. However, we still need to 
get the remaining departments to develop their matrices and, perhaps more importantly, 
we need to develop a means of auditing programs to ensure that the proposed curriculum 
is being taught and assessed in the manner described. This coming year, the Faculty 
Senate Assessment Council will perform departmental assessment audits as part of regular 
internal program reviews (approximately five to six per year) to determine if faculty are in 
fact utilizing teaching and assessment methods appropriate for the program's stated 
outcomes. A similar auditing process must be developed for university studies. The fact 
that all course proposals to the revised university studies program must include a 
description of how the outcomes are to be assessed and how the information from those 
in-class assessments is to be used to improve teaching and student learning is a major 
accomplishment. 

GOAL 5. Pedagogy. 

Results. Approximately 60% of Central's faculty have added assessment-as-learning 
exercises to their classes that are driven by general or program outcomes. Refer again to 
Exhibit 3 which describes the results of student and faculty surveys designed to measure 
changes in faculty and student perceptions and activities due to CPI. 

Evaluation. More work needs to be done in both faculty development and administrative 
support of teaching in order for us to get the 80 to 85% compliance we feel is necessary 
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to succeed. We have not made enough progress in changing the reward system to truly 
emphasize excellence in teaching as a prerequisite for promotion and/or merit pay. 

GOALS 6 & 7. Training/Faculty 

Results. In the first three years of the project. Central held five university forums, two 
university planning retreats, and ten days of workshops for faculty. In addition to these 
centrally organized training sessions, most of the CPI departments established monthly 
meetings to plan and evaluate their progress towards the ten components. Over the last 
four years, approximately 70% of Central's faculty have received some type of training in 
CPI. A significant proportion (10-15%) have received extensive training in assessment-as- 
learning and/or total quality management. Sixty-seven of Central's faculty have attended a 
five day workshop on assessment-as-learning at Alverno College. Each year we have held 
a university wide two-day workshop on a variety of topics related to implementation of 
CPI. This past summer (June 16-17, 1995) the workshop focused on preparing faculty to 
teach in the revised university studies program. Over 85 faculty learned how to develop 
assessment-as-learning exercises for their classes and how to use the information from 
those assessments to improve teaching and learning. Three more workshops have been 
scheduled for the remainder of the year. As mentioned above, new faculty will attend a 
workshop on CPI as part of their formal orientation program. 

Last fall, four faculty Outcomes Teams, based on the four general outcomes, were formed. 
These teams are the implementation arm of the University Studies Committee. The faculty 
teams directed monthly meetings for faculty (30-35) who were interested in developing 
expertise in performance-based assessment. By next fall, all faculty scheduled to teach in 
the revised university studies program should have received initial training in performance- 
based assessment. 

During the course of the CPI project, over 200 faculty have collaborated with colleagues, 
alumni, employers, and professionals to define and validate outcomes in their majors, 
evaluate and revise curricular expectations, and develop outcomes assessments. 

Evaluation. For the last three years we have attempted to add five days for faculty 
training and/or departmental planning to the academic calendar. Our goal is to eventually 
have two weeks set aside each year for planning and development. Training remains a 
high priority. 

All training sessions organized and managed by the CPI team were formally evaluated as to 
process and product. An example is provided in Exhibit 1 1 . The results of these 
evaluations were made public and were used to plan higher quality workshops in keeping 
with the principles of the CPI model. 

From the outset we seriously underestimated the importance of faculty training and 
development. We did not anticipate either the amount that was needed or the difficulty in 
getting faculty to attend training sessions. Since the end of the first year (1992) we have 
placed tremendous emphasis on faculty development and are convinced that systems 
reform is impossible without it. We have made great strides in improving both the 
involvement in and attitude toward training on the part of our faculty. 
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GOAL 8. Students. 

Results. Over 5,400 students have been exposed to CPI. Over 600 students attended 
special workshops developed by the project team. This fall, a team of students, sponsored 
by a faculty person in the Communication Department, will develop a CPI advertising 
campaign that will be used on and off campus to inform students and the public about 
CPI. It is hoped that this campaign will increase campus wide awareness of CPI 
substantially. 

Evaluation. Our goal to educate students about CPI has been relatively successful as 
indicated by both campus wide and departmental student surveys, and student attendance 
at various workshops. However, our objective of being able to document increased 
student learning, and graduates who are better able to meet the demands of professional 
and graduate schools, employers, and the private and public sectors, has not been 
achieved. It will take approximately another three years before we have the CPI quality 
system in place to the degree we can collect comprehensive data on students relative to 
either the general or program outcomes. 

GOAL 9. Co-Curricular. 

Results. Approximately 600 students have participated in Central's co-curricular transcript 
program since the beginning of the CPI project. 

Evaluation. Due to the fact that the CPI model has not yet been fully implemented, we 
have not been able to develop the co-curricular component to any significant degree. At 
this point in time we have mostly identified co-curricular offices and activities that can 
provide opportunities for relevant, experiential learning exercises for our students. We 
have not actually developed co-curricular exercises and assessments to the degree desired 
or possible. Currently, only field experiences (e.g., internships, practicums) in a few of 
our programs include systematic learning and assessment experiences guided by CPI 
principles. 

GOAL 10. Dissemination. 

Results. Over 120 presentations to state, regional, and national audiences have been 
made to date. Exhibit 12 provides a listing for just the project director. In addition, four 
major papers, thirty proceedings, five workshops at other institutions, and over 125 
responses to inquiries have been performed. A cooperative learning venture with eight 
other institutions has been agreed upon and submitted to FIPSE as the basis of a 
dissemination grant. 



F. Summary and Conclusions . 

A paradigm shift has begun in higher education. Institutions on the leading edge will 
gain an advantage--just as in business. The continuous process improvement (CPI) model 
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is one viable alternative for reform. At least four reasons exist to draw from CPI as a 
guide to reform. 

First, the CPI model strikes at the heart of the serious flaws in the course credit 
completion model. Since this paradigm powerfully influences all curricular and pedagogical 
decisions, a precondition for fundamental reform is to draw it into consciousness and 
examine it. The CPI model does this. 

Second, the CPI model focuses on system versus individual improvement where W. 
Edwards Deming (1960) insists 85 percent of improvement potential exists. 

Third, the CPI model as an alternative paradigm forces the educational process to be 
responsive to the ever changing needs of society, because it requires explicit definition of 
student performance outcomes as defined by both internal and external customers 
(faculty, students, alumni, employers, etc.). These outcomes provide an effective means 
coupled with extensive in-class and comprehensive assessment feedback for continuous 
learning and improvement--a fundamental assumption of assessment as learning and 
principle of TQM. The course credit model provides no comparable mechanism to drive 
continuous improvement. 

Fourth, Central's CPI project, initiated and developed by faculty, is demonstrating that 
faculty will provide leadership for substantive improvement. The model assures, once 
accepted by faculty, their full interactive involvement. With faculty leadership and 
involvement, real reform that reaches into the classroom and affects student learning will 
happen, resulting in long-term change. 

Central has sucessfully (1) piloted the CPI model in 19 departments and 25 programs 
(Phase IV); (2) secured commitment of faculty and administrators to continue to fully 
implement Phase V; (3) has won accolades for the effacacy of the CPI model from national 
leaders (Exhibit 1 3), and (4) has developed the plan and commitments for adaptation to 
eight potential benchmark institutions setting the necessary condition for a empirical test 
of the two models -- course-credit and continuous process. 
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Assessment-As-Learning Assumptions 



Exhibit 2b 



stm-tent Learning and Assessment 

1. Student learning is a primary purpose of an educational institution. 

2 Education goes beyond knowing to being able to do what one knows. 

3. Learning must be active and collaborative. 

4. Assessment is integral to learning. 

5 Abilities must be developed and assessed in multiple modes and contexts. 

6 Performance assessment— with explicit criteria, feedback and self assessment-is 
an effective strategy for ability-based, student-centered education. 

Curricular Coherence and De velopment 

7 A coherent curriculum calls for faculty investment in a community of learning and 

judgment. .... . 

8. The process of implementation and institutionalization of a curriculum is as 
important as the curriculum; the process is dynamic, iterative, and continuous. 

9. Educators are responsible for making learning more available by articulating 
outcomes and making them public. 

10 Responsibility for education involves assessing student outcomes, documenting 
inputs, and relating student performance over time to the curriculum. 



From the Consortium of Teaching, Learning and Assessment, funded by W.K. Kellogg Foundation, 
including Alverno College, Bloomfield Hills Model High School, Central Missour, State Un.versity 
Clayton State College, Gallaudet University, Macomb Community College, Purdue University School 
of Pharmacy, South Division High School, Township High School District #214, University of 
Wisconsin Medical School, University of New Mexico School of Medicine. 
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Exhibit 2c 

Key Principles of TQM 

TQM is a set of management principles and core values. While each of the founders of 

TQM (Deming, Juran, Crosby) define their essentials differently, there are common 

themes. We focus on four: 1) Customer focus; 2) Commitment to Continuous Process 

Improvement; 3) Involvement; and, 4) Systems thinking. 

1. Customer focus: Customer satisfaction defines quality. Just as the value of a gift is 
defined by the receiver, quality is defined by the customer. Everyone in the system 
must identify and develop a working relationship with their customers. External 
customers are the receivers of your system's product or service. Internal customers 
are within the system e.g., students, the faculty instructor next in line who is 
dependent on what the student learns in your class. Internal customers are important 
in defining, assessing, and improving the process. Everyone should identify, define, 
measure, and meet the criteria for satisfaction of their customers, this begins when 
people make a deliberate effort to identify their internal and external customers. It 
matures when all customer-supplier relationships are well defined partnerships. 

2. Commitment to Continuous Process Improvement: Everything is a process. TQM 
focuses on how each process can be improved. Continuous improvement assumes 
well-defined objectives, criteria, and measurement (assessment). This requires a deep 
personal and shared commitment to quality (excellence) which transcends other 
personal and short-term concerns; thus, the commitment is by nature enduring and 
strategic. This correctly implies the necessity of a fundamental cultural change in 
many organizations. 

3. Total Involvement: Involvement goes beyond many earlier participatory management 
notions. It means more than encouraging cooperation, sharing responsibility, 
participation in some decision making, and working in teams. Involvement is 
facilitated by providing quality education and training initiatives which allow 
employees to learn and use skills which go beyond current job task requirements in 
order to redesign work processes. Involvement may be the key to simplifying 
processes. Involvement assumes everyone is a valued and competent partner who 
believes in and acts on the ideal of quality. 

4. System Thinking: TQM asserts 85 percent of total error is "common cause variation" 
or "system error," only 15 percent results from individual performance. TQM is 
fundamentally different (paradigm shift) from traditional management which may be 
inordinately concerned with individual performance. 
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Assessment 



Prior to President Elliott’s arrival in 1985, there were no institution- wide assessment 
efforts at Central. Upon his arrival, however, the President quickly stressed the importance of 
being accountable to the public for what we do, and that assessment plays a key role in 
accountability. For the past decade, President Elliott has continued to emphasize the 
importance of assessment, and has provided the administrative leadership and support, as well 
as the financial resources, for the university to make progress in this arena. 

Assessment Coordinator 

In 1986, the position of Assessment Coordinator was created. The Coordinator was 
charged with responsibility for developing and implementing a comprehensive assessment 
program in accordance with Central’s Mission Statement. Other duties of the Coordinator 
include administering the assessment budget, serving as a liaison between the administration 
and the Faculty Senate Assessment Committee, assisting the Director of Institutional Research 
by providing assessment data for institutional reports, and preparing an annual assessment 
report. Dr. Jim Sylwester, Professor of History, was hired as Central’s first Assessment 
Coordinator. Dr. Mike Grelle, Professor of Psychology, was hired as Central’s second 
Assessment Coordinator in August of 1993. 

Assessment Efforts 

Although there were no university-wide coordinated assessment efforts prior to the 
creation and implementation of Central’s Continuous Process Improvement model (see page 
46A), various assessment efforts have been made at the college and department levels 
throughout the past decade. 

1985 - 86 

• Senior-level teacher education students majoring in the arts and sciences participated 
in the National Teacher Examinations. The College of Arts & Sciences recommended 
that a system of proficiency testing be adopted institutionally. 

• The College of Business & Economics developed an advisory council, to be 
implemented during the 1986-87 academic year. The council was composed of top 
executives from major companies who were employers or potential employers of 
university graduates. 

1986 - 87 

• Faculty evaluation was implemented in every department during 1986-87, and 
instruments were created for the evaluation of faculty members and department chairs. 



• Assessment was a primary focus of the College of Applied Sciences & Technology in 
1986, and although some departments like the Department of Nursing already had 
extensive assessment processes in place, very few such programs existed on a national 
level in the technology disciplines. Therefore, work was begun to develop assessment 
techniques for other technology programs from within. 

• Important progress was made by the College of Arts & Sciences in the areas of 
writing and mathematics assessment, including proposals to implement a Writing 
Across the Curriculum program. 

• The College of Business & Economics gave substantial effort to the establishment of 
its advisory board. The College also developed an assessment plan for business & 
economics majors to be implemented in the fall of 1987. 

1987-88 

• Continued progress was made in mathematics and writing assessment. 

• The Department of English demonstrated leadership in writing assessment by hosting 
a conference at which all the Missouri universities developed plans to collaborate in 
writing assessment efforts. 

• The Department of Agriculture held a cooperative conference for all agriculture 
programs in Missouri, the purpose of which was to consider cooperation for the 
development of joint assessment instruments. 

• The College of Business & Economics developed an overall plan for assessment 
within the college, and utilized a number of assessment instruments during the year. 
The college also demonstrated leadership in the area of teaching assessment through 
selective use of the Kansas State IDEA System, a tool used for collecting student 
opinions on the effectiveness of teaching. 

• The College of Education & Human Services was involved in assessment through the 
administration of the National Teachers Examination and the College-Base 
examination. 

• Thirteen departments throughout the university participated in assessment of their 
majors by working with the Educational Testing Service in an experimental program. 

• Under the direction of a new Dean of Library Services, a major self-study of 
Educational Media Services was completed. The resulting recommendation was that 
EMS become a part of the library. 
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1988 - 89 

• The departments of music and theatre developed sophisticated assessment processes. 

• The College of Business & Economics created an Assessment Committee, and the 
college’s assessment requirements and strategies were evaluated in depth. 

1989 - 90 

• A five-year review of assessment activities at Central highlighted current efforts: use 
of student scores on national standardized tests and professional licensing exams for 
placement purposes; assessment tests given during the course of student programs; use 
of alumni surveys, employer surveys, and student and faculty satisfaction inventories. 

1990 - 91 

• The IDEA System was broadly implemented by the College of Business & 

Economics, and in areas of the College of Applied Sciences & Technology. 

• Four members of the Department of Curriculum & Instruction developed a 
"teacherness" assessment instrument, in cooperation with Selection Research, Inc., and 
six other institutions of higher education. 

• A major, 10-department effort to develop a "Continuous Process Improvement" (CPI) 
model for the assessment of teaching and learning began, resulting in a proposal for 
funding by the federal Fund for Improvement of Post-Secondary Education (FIPSE). 

• Work progressed with Alverno College on a Kellogg Foundation-funded project in 
assessment and the improvement of learning. 

• The founder of the Harvard Assessment Seminars visited Central to share information 
about Harvard’s program. 

v 

• Faculty members attended national meetings to sharpen assessment knowledge and 
practices. 

1991 - 92 

• The Continuous Process Improvement model was implemented in 10 academic 
departments and 13 individual programs across the university. 

• Assessment was added as a seventh standing faculty governance committee in the 
College of Education & Human Services. 

1992 - 93 

• Expansion of CPI model and project 

• Library Services established an assessment process to determine effectiveness of 
services, client and employee satisfaction, and to target areas for improvement. 
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1993-94 

• The Department of Agriculture established an assessment plan for all bachelor’s 
degree programs, and refined lists of competencies. 

• The College of Arts & Sciences refined departmental assessment plans. 

• The College of Business & Economics made progress in several areas of assessment: 
CPI was used as the basis for program design, assessment and continuous 
improvement; the College also focused on measurement of program and student 
performance relative to program and student outcomes; curriculum and courses were 
revised to reflect feedback from program and student performance measurements and 
from consultation with external constituents; in addition, graduates’ success was 
assessed through contact with alumni and employers. 

• The College of Education & Human Services addressed assessment in departments 
and in several centers. 

• Graduate Studies & Research established assessment and multicultural teaching 
criteria for annual graduate program review. 

Continuous Process Improvement (CPI) Model & Project 

During the 1987-88 school year, a core group of faculty members at Central began 
working toward the development of an institutional assessment plan. The group sought 
answers regarding WHAT should be assessed, WHY it should be assessed, and finally, a 
means or model that would facilitate such assessment. Faculty members determined that the 
WHAT should be student learning, and the WHY was because student learning is primarily 
valued. The group continued to search for a model and, in June of 1988, discovered Alvemo 
College’s assessment-as-learning model. In 1990, the group realized the congruence of that 
model with total quality management principles. 

In September 1991, Central was awarded a three-year grant in the amount of $238,927 
from the Fund for the Improvement of PostSecondary Education (FIPSE). The FIPSE project 
called for Central to (1) explicitly define a process model from the two sets of principles, 

(2) specify and secure faculty acceptance of a clear set of goals, and (3) develop a set of step- 
by-step process components to enable faculty to incrementally achieve the goals. Central 
committed to FIPSE to design, introduce, and implement the Continuous Process 
Improvement (CPI) model in 10 academic departments and 13 major programs in the first 
year. Such implementation occurred in the first year of the grant, 1991-92. By 1994-95, 

17 departments and 26 major programs were directly involved in implementation of the CPI 
model. 
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Presently, members of the Central community are working to fully implement and 
integrate CPI into the university culture. President Elliott has shown continuous support for 
the model, and acknowledges CPI as Central’s unique learning system in communications 
with faculty, parents, students, alumni, and the public. The academic deans have committed 
to complete implementation of the CPI model in all 34 academic departments by May 1998. 

In addition, Central’s Board of Regents approved the transfer of $250,000 from the 
university’s quasi-endowment fund to stimulate CPI development in 1995-96. A table on the 
following page outlines the implementation plan for Central’s CPI model. 

TQM/TQE 

In addition to initiatives encompassed by the CPI model and project, there have been 
numerous other efforts made at Central in the movement toward a total quality environment. 

In non-academic areas, the use of cross-functional teams is increasing. Cross-functional teams 
are teams which are comprised of members from all facets of the campus - faculty, staff, 
students, administrators - and function specifically to examine, improve and/or refine current 
processes and procedures at the institution. Examples of cross-functional teams now in 
existence include the Quality Self-Assessment Team, the Enrollment Management Team, and 
the Start-Up Committee. The use of advisory boards and committees is also increasing. The 
Fee Payment Advisory Committee, the Commencement Committee, and the Traffic and 
Parking Policy Committee are all examples of advisory boards/committees which are currently 
utilized in non-academic areas at the university. 

In Fall 1994, Craig Christie was employed by the university to serve as a total quality 
consultant. Christie brought extensive experience from the business sector regarding how to 
transform the institutional culture and environment into one of total quality. Christie’s 
expertise was shared during workshop sessions with staff members in the Library, and with 
faculty and staff members in the College of Business & Economics. Christie also addressed 
numerous courses in the College of Business and Economics regarding total quality concepts 
and principles. He was retained by the university through spring of 1995. 

In the spring of 1995, Central’s Quality Self-Assessment Team (QSAT) recommended 
that the campus pursue a self-assessment of current quality initiatives. The QSAT also 
recommended that the self-assessment be based on the criteria defined by the Malcolm 
Baldridge National Quality Award and the Missouri Quality Award programs for the 
Education Category: Leadership; Information and Analysis; Strategic and Operational 
Planning; Human Resource Development and Management; Educational and Business Process 
Management; School Performance Results; and Student Focus and Student and Stakeholder 
Satisfaction. The quality self-assessment process was initiated on campus in the summer of 
1995 and will be completed by spring 1996. Based on the findings of the QSAT, Central 
may apply for a Missouri Quality Award. 



Assessment Philosophy 



Central Missouri State University adopted an assessment philosophy in spring of 1 992. 

The philosophy states that. 

Assessment is an integral part of the continuous process of learning and 
development with the purpose of enhancing a student’s total university 
experience. Assessment uses well-defined outcomes and criteria and multiple 
measures. The information derived from assessment activities will be used to 
facilitate student learning and development, to promote faculty and staff 
growth and to improve the quality of academic and nonacademic programs, 
services and facilities. 



Academic Assessment Paradigm 



The following Academic Assessment Paradigm was also adopted by the university in 
spring of 1992. 

Academic assessment at Central Missouri State University depends upon faculty and 
student involvement and occurs in the form of designing, monitoring, administering 
and evaluating assessment activities within departments. University Studies and 
interdepartmental programs. Academic assessment at Central Missouri State 
University: 

• is directly related to teaching 

• is directly related to learning 

• provides information for program improvement 

• is to be used in a positive manner 

Student Services Assessment Paradigm 



Central’s Student Services Paradigm states that Student Services Assessment at Central 
Missouri State University: 

• contributes directly to student learning and development 

• relates directly to providing support services for students and constituents 

• uses multiple means to gather information for program improvement 

• recognizes that students often are experiencing rapid personal growth 

• recognizes that "adult learners" are an important segment of the university 
population 

• involves the interaction of staff and students outside of the traditional classroom 

• fosters personal growth in independence and leadership 

• perceives that human diversity is a strength 

Central’s Student Services Assessment Paradigm was adopted in fall 1993. 
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Exhibit 6 










Sage Publications, Inc. 



Managerial Skills 

9 A Study of the Assessment Center Method of 
Teaching Basic Management Skills 



Ralph F. Mullin 
Paul L. Shaffer 
Michael J. Grelle 



Phase I: A Pretest-Posttest of Assessment Center Impact 

Thirty years ago, Gordon and Howell (1959, pp. 44,81, 104-105) identi- 
fied four basic sets of skills purported to make up business competence. 
They found these skills, necessary for success in business, were clearly 
not being developed by business schools. 

Neither their published statements, their educational programs, nor our 
conversations with deans and faculties in all parts of the country revealed a 
clear awareness of what these skills arc or how they may best be developed 
(p 45). 

In 1983 Cameron and Whettcn (1983) reviewed a decade of criticism 
of business school curricula and concluded that management skills were 



AUTHORS’ NO ITS: Ral|4i 1 : . Mullin and Paul L. Shaffer wrote Phase I of this chapter, 
Michael J Grelle joined them in the writing of Phase D. 

We grateful ly acknowledge support for this research from Central Missouri State University, 
which funded the project ($75,000) to provide a research grounded basis for development 
of a model of learning and assessment that will focus on explicit performance ‘based student 

outcomes. 

Reprinted by permission o! Sage Publications, Inc 
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still not being taught — at best, students arc taught about management 
(content knowledge) rather than how to manage. The American Assembly 
of Collegiate Schools of Business (AACSB) Outcome Measurement 
Project Report explicitly recognizes that basic management skills are 
“seldom systematically addressed by curricula” (1987, p. 2). Porter and 
McKibbin’s (1988, pp. 71-72) findings continue to confirm the rather 
large gap between the current reality and what “should” be in terms of 
behavioral emphasis in business school curricula. The “Big 8” accounting 
firms’ Perspectives on Education (Accounting Education Change Com- 
mission,1989) slates: 



There must be a focus on ihc broader skills. . . . Without a clear set of 
capabilities to use as objectives in the curriculum design process, it is 
unlikely that changes in the current content or teaching methods will be 
responsive to the needs of the profession. . . . Basing pre-entry education 
(university) on capabilities will mean fundamental changes in the curricu- 
lum. (p. 5) 

In spite of the evidence that basic management skills are not being 
systematically integrated into B-school curricula, they arc frequently a 
topic at professional conferences. 

What are these basic management skills? The AACSB Outcome Mea- 
surement Project (1987) identifies and defines both academic subject 
knowledge (seven content areas) and skills and personal characteristics 
(SAPCs). Development Dimensions International (DDI), in the skills 
diagnostic program (SDP) designed for AACSB, elaborates nine dimen- 
sions of management skills and personal characteristics (SAPCs). Impor- 
tantly, DDI has developed operational definitions and a set of assessment 
center type exercises and evaluation procedures to measure these skills. 
While there are numerous taxonomies of the basic skills essential to 
successful practice, the degree of commonality is indeed striking. 
Table 9.1 displays the similarities of a few of these. 

How can these skills best be developed? Cameron and Whcttcn (1983) 
emphasize that achieving change in student management skill behaviors 
requires a different type of learning than docs the acquisition of content 
knowledge. The chief executive officers of the “Big 8," in Perspectives 
on Education, suggest new methods arc demanded. 

The current icxibook-bascd, rulc-inicnsivc, Iccturc/problcm style should not 
survive as the primary means of presentation. New methods, both those used 
in other disciplines and those that arc totally new to university education, 
must be explored. (1989, p. 1 1) 
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Assessing Managerial Skills 



Gordon and Howell ( 1 959) suggest three emphases in teaching business 
courses — descriptive, analytical, and managerial -clinical. They repeat- 
edly stress "managerial-clinical teaching" as particularly well suited for 
teaching managerial skills (pp. 109, 135, 136, 360). Porter & McKibbin 
repeat the emphasis on the "clinical approach” (1988, p. 342). The assess- 
ment center method is one operationalization of the clinical method, as it 
entails one-on-one expert observation of the learner’s practice perfor- 
mance with developmental feedback and repeated performance for 
improvement Alvemo College’s application of llic assessment center 
method to student learning, called asscssmcnt-as-lcaming, is based on the 
principle that students will learn developmental^ by continuous assess- 
ment of performance plus feedback (Marchcsc, 1987). Tins performance- 
based learning process is explained by Bandura’s social learning theory 
(1977). Table 9.2 extends Cameron and Whetten’s (1983) model to more 
completely operationalize Bandura’s theory. 

A number of business schools have experimented with teaching man- 
agement skills. These include Alvemo College, Boise State University, 
Brigham Young University, The University of Illinois, University of 
Pittsburgh, Utah Stale University, and Detroit University. This study is, 
therefore, viewed as an extension of this earlier work and as a pilot study 
preliminarily testing the effectiveness of the assessment center method for 
student learning of the set of management skills identified by AACSB and 
operationalized by DDI. Ed Pavur of DDI (1988) comments on the 
uniqueness of the study: “While SDP assessments have been conducted 
in schools across the country since 1983, your use of the program is 
unique. No other application has the set of characteristics which you arc 
employing.” 

Methodology 

The general hypothesis is: Students who participate in the experimental 
course will demonstrate significantly higher levels of competency on a 
set of basic management skills as measured by equivalent DDI pre- and 
postasscssmcnt exercises. 

Sample 

Thirty-five junior, senior, and graduate business students registered in 
a course tided Business Administration Practicum, offered as an elective 
in die fall of 1988. Although subjects self-selected into the sample, an 
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evaluation and scoring. Data on the reliability and validity of these 
exercises arc reported in AACSB (1987). The posttest (02) measured the 
35 students on eight of the original nine dimensions, using DDI parallel 
exercises. The posttest instruments and external scoring were also pro- 
vided by DDI. 

The post-training simulations can be considered parallel to the pre-training 
simulations because of overlap in the key behaviors and dimensions which 
the simulations elicit. Specifically, 92% of the key behaviors and 100% of 
the dimensions are targeted for observation in both of the in-basket simula- 
tions; 100% of the key behaviors and 100% of the dimensions arc targeted 
for observation in both of the interview simulations. ... the two sets of 
exercises represent parallel versions and should meet your needs for equiv- 
alent materials. (Pavur, 1988) 

In addition to the DDI skill performance measures, two questionnaires 
were administered to the 35 students — at the first and last sessions of the 
course. These coveted demographics on school grade level (GLVL) and 
grades (GPA), parental occupation (POCC) and education (PED), work 
experiences (EXP), plus students’ self-report perceptions of course impor- 
tance (CIMP), course relevance (CREL), commitment to study (COMS), 
and level of performance (PERF) for each of the skill dimensions (begin- 
ning and ending). 

The Learning Intervention 

* 

The learning intervention (X) consisted of: individualized conferences 
for feedback of pretest results and initial goal planning; ten 3-hour 
learning modules emphasizing team and individual experiences in prac- 
ticing and assessing management skill exercises; and evolution of each 
student’s goals and development plan. Table 9.5 oudines the course 
content and schedule. Resources were provided by the university and the 
participating telecommunications firm. 

Methods of Analysis 

To test the general hypothesis, both MANOVA and /-tests were per- 
formed to determine if there was a significant difference between pretest 
and posttest scores on measures of the eight dimensions. The MANOVA 
analysis of the pre- and postircaimcnt results was approached as a 2 x 8 
within-subjccts design. 
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Tabic 9.4 

Measures: DDI Set of Assessment Instruments 



1 . In-Basket Exercise 

Role: Middle-level manager, manufacturing 

Task: Managing a budget, directing subordinates, and carrying out project planning, 
managing labor, public Sc government relations 
Presentation: Response in writing to the in-basket items 

Dimensions: Planning Sc organizing. Judgment, Problem analysis. Delegation, Control, 
Written communication 
Time: 3 hours, 15 minutes 

2. Analysis/Oral Presentation Exercise 
Role: manager 

Task: Make recommendations to senior management re. expansion of production 
capacity, financing, product mix 

Presentation: Written report Sc brief oral presentation — videotaped 

Dimensions: Judgment, Problem analysis. Oral presentation. Written communication 

Time: 3 hours, 15 minutes 

3. Planning Exercise 

Role: Manager, company finance division 
Task: Plan start-up of a word -processing unit 
Presentation: Written planning document 

Dimensions Assessed: Planning and organizing. Written communication 
Time: 2 hours 

4. Group Discussion Exercise 

Role: Middle-level manager, large organization 

Task: Represent department and candidate for share of $8,000 salary increase funds 
(discretionary) 

Objective: Secure maximum share for candidate and complete group task quickly 
and fairly 

Presentation: Group discussion and data videotaped. 

Dimensions: Oral communication. Group leadership. Oral presentation 
Time: 2 hours 

5. interview Simulation 

Role: Manager, Thrift Program Management section of a large bank 
Task: Business development and administration of thrift savings programs which 
corporations offer to employees as part of a benefits package 
Objective: Meet with employee whose performance has slumped recently and help 
him/her solve problems and improve performance 
Presentation: Videotaped interaction with trained role player 
Dimensions: Control, Oral communication. Individual leadership 
Time: 1 hour 

6. Self-Report Inventories 
Instrument: Two self-report inventories 
Dimension: Disposition to lead 

Time: Administration is untimed, but should require no more than 1 hour and 30 minutes 
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Tabic 9.4 

Measures: DDI Scl of Asscssmcnl Instruments 
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Presentation: Response in writing to the in-basket items 

Dimensions: Planning & organizing. Judgment, Problem analysis. Delegation, Control, 
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Time: 3 hours, 15 minutes 
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Task: Make recommendations to senior management re. expansion of production 
capacity, financing, product mix 

Presentation: Written report & brief oral presentation — videotaped. 

Dimensions: Judgment, Problem analysis. Oral presentation. Written communication 
Time: 3 hours, 15 minutes 

3. Planning Exercise 

Role: Manager, company finance division 
Task: Plan start-up of a word-processing unit 
Presentation: Written planning document 

Dimensions Assessed: Planning and organizing. Written communication 
Time: 2 hours 

4. Group Discussion Exercise 

Role: Middle-level manager, large organization 

Task: Represent department and candidate for share of $8,000 salary increase funds 
(discretionary) 

Objective: Secure maximum share for candidate and complete group task quickly 
and fairly 

Presentation: Group discussion and data videotaped. 

Dimensions: Oral communication. Group leadership. Oral presentation 
Time: 2 hours 

5. Interview Simulation 

Role: Manager, Thrift Program Management section of a large bank 
Task: Business development and administration of thrift savings programs which 
corporations offer to employees as part of a benefits package 
Objective: Meet with employee whose performance has slumped recently and help 
him/her solve problems and improve performance 
Presentation: Videotaped interaction with trained role player 
Dimensions: Control, Oral communication. Individual leadership 
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6. Self-Report Inventories 
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Dimension: Disposition to lead 
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Tabic 9.5 

Managerial Skills Assessment and Development: Course Format and Schedule 



Module * 
it Date 


Acfivi/y <Sl Method 


Dimension! Exercise 


Assignment 


1. 8/24- 
9/3 


PRETEST 13 hours of 
assessment exercises 


All six DDD Exercises 
Eleven Dimensions 


None 


2. 10/3 
10/4 


Feedback DDI Results 
Team Assignments 
Assess Performance 
Assess Improving Perf 


All All 

Interview: 3 L,OC,C 
Interview: 


Self-assess 

Interview Ex 
Video tape 


3. 10/10 
10/11 


Discuss Course Plan 
Rate 2 training Videos 
Improving Work Habits 
Delegating Resp. 

Group Discussion 


FB on Dimensions 
Interview 

Delegating 


Work pp 193*202 
Module HI 
Phase I 


4. 10/17 


Leadership Concepts 
Values Clarification 
Goal -sett ing/Planning 


Self-analysis Quest 
Values Exercise 
Goal-setting Ex. 


Do Module 1 
parts 1-4 
Meet w Mullin 


5. 10/24 
10/35 


Goal-setting 
Time Management 
Decision-making 


Finish Goals Draft 
7 Day Time Analysis 
Exercise, p.263 


Mod. I Part 4 
Mod. I Part 6 
Mod HI Part 5 


6. 10/31 
11.1 


Review Interview Ex. 
Interview Exercise 
Team Evaluation of Ex. 


Ixadership/Conlrol 
/Oral communication 


Review Interview 
Exercise Guide 


7. 11/7 
11/8 


Complete In-basket ' 
Exercise (Woodlands) 


ln-baskct, 6 Dmns 
3 hour exercise 


Review of In- 
basket Guide 


8. 11/14 
11/15 


Assess In-basket Ex. 
Terry Butler 
(Union 152-4) 


PI an n in g/or g a ni za in g 
Judgment, analysis 
Control, Delegation 


Self-assess 
DDI In-basket 


9. 11/21 
11/22 


Assess In-basket Excr. 
(Union 152-154 


l^aura Marquart, UTS 
Malloy Gould, CMSU 


Complete UTS 
In-basket Ex. 


10. 11/28 
11/29 


POST-TEST 

DDI Interview Exercise 


Three Dimensions 


Review videos 


11. 12/5 
12/6 


DDI In-basket Exercise 
(Union 152-154) 


Six Dimensions 


Review: 3 In- 
baskets + notes 


12. 12/9 F 
4 -6PM 


Focus Groups 


Course Evaluation 
Term Paper due 


Review notes 
Complete Goals 



* Three-hour class except for the 13 hours of pretesting 
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Tabic 9.6 

Test of General Hypothesis 



Learning Effects (Difference) 



MANOVA (2x8) 


Canonical Cor. 


p-valuc 


Pre-Post Difference 


.97 


0.000 


Interaction: 


.82 


0.001 


Pre-Post (2) with Dimensions (8) 

Mests: Difference (Posttest — Pretest) on Individual Dimensions 




Dimension 


f-valuc 


p-valuc 


1. Leadership 


1.43 


0.16 


2. Oral Comm. 


-0.48 


0.63 


3. Written Comm. 


-0.30 


0.77 


4. Planning & Org 


0.91 


0.37 


5. Analysis 


1.77 


0.08 


6. Judgment 


3.33 


0.002 * 


7. Delegation 


6.35 


0.000 * 


8. Control 


“0.15 


0.88 



NOTE: In addition to tests of the general hypothesis, several analyses were performed to identify 
independent variables that predicted performance for pretest (TRET), posttest (POST), and gain (GAIN) 
scores. A series of regression analyses (including stepwise) were used to determine the significance and 
magnitude of these effects. Considering the small sample size, a priori statistical significance was set at 
a p- value of 0.05. 



Results 

The general hypothesis was supported. Both a /-test on total gain scores 
for all eight dimensions (p = .0001) and the MANOVA 2x8 (Canonical 
Correlation = 0.968, p - .0001) provide statistically significant evidence 
that learning from the pretest and treatment was effective in increasing 
overall student performance on the target set of basic skills as measured 
by the pretest- posttest difference (GAIN). 

It was not hypothesized that the students would show statistically 
significant (p = 0.05 or less) gain scores on the eight individual skill 
dimensions; however, gains in two of the eight dimensions — judgment 
and delegation — were significant. Additionally, the analysis dimension 
approached significance (Table 9.6). 
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Discussion 

The results provide evidence tiiat students can leant basic management 
skills in an academic setting using an assessment center method that 
combines skill practice with assessment and feedback. The one-group 
pre-post research design, though commonly used in educational research, 
is weak from a research perspective. This design has several obvious 
inadequacies, for example, measurement, maturation, history, and regres- 
sion effects. The most serious of these, in terms of affecting dependent 
variables, is measurement effect. In this ease, the pretest measure is highly 
reactive — 13 hours of assessment exercises. Used in this manner the 
pretest is an important and integral part of the learning treatment itself. 
Measurement of the independent learning effects of the pretest and 
intervention will require a stronger research design. 

Adoption of the assessment center method for management skill assess- 
ment and development by business schools will require convincing evi- 
dence that the assessment center method and specific operationalizations 
produce significantly greater learning outcomes than traditional methods. 
Thus, a study using a stronger research design and longitudinal tests 
should be conducted. 

A more difficult question for research is whether Uicsc skills arc learned 
more effectively within specially designed courses focused on Uicsc skills 
or by systematic integration of skill learning and practice across the 
curriculum. For example, testing the comparative effectiveness of the 
University of Illinois program, where basic skills arc taught in a dedicated 
Principles of Management course, and Alvemo College’s program, where 
skill assessment and learning are integrated throughout die curriculum. 



Conclusion 

The literature suggests dial the need for business schools to teach basic 
management skills has been evident for 30 years. Business schools as a 
group have, consciously or unconsciously, failed to make a well-defined 
response. Results of die preliminary study support the proposition dial 
basic management skills, as defined, can be learned in an academic 
setting, using the assessment center mcdiod. The assessment center 
mcdiod may provide business school faculty and administrators a credible 
means for response. Furdicr study, however, is needed to determine the 
differential effects of alternative mcdiods. 
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Phase II: A Solomon Four Design Test of Methods 

This section extends the study described previously, which provided 
evidence that students can learn a well-defined set of basic management 
skills in a traditional undergraduate academic setting, using the assess- 
ment center method that combines skill practice with assessment and 
feedback. The preliminary' study’s limited one-group pre-post research 
design, however, provided no evidence as to the comparative effective- 
ness of this method. 

Since adoption of the assessment center method for management skill 
learning by business schools will require more convincing evidence, one 
purpose of this Part 2 study was to test the effectiveness of the assessment 
center method against a more traditional textbook/lecturc-intcnsivc 
method, using a stronger research design. Additional research questions 
included: Can learning of basic management skills be achieved without a 
commensurate loss of content knowledge learning? Will faculty using the 
assessment center method receive lower student course evaluations? Can 
basic management skills be adequately learned within a single course 
(c.g., Whctlcn [19XX], University of Illinois), or does skill acquisition 
require student learning and assessment developmental^ across the cur- 
riculum (c.g., Alvcmo College’s ability-based learning program)? The 
following hypotheses were developed to test all but the last question: 

HI: Students who participate in experimental, assessment-center method 
classes (XI) will demonstrate higher levels of competence on a target set 
of basic management skills, measured by equivalent DDI prcasscssmcnts 
(01) and poslasscssmcnts (02), titan students in traditional classes (X2); 

112: Students who take the DDI skills prcasscssmcnt (01) will perform higher 
on the parallel poslasscssmcnts (02) titan students not given the extensive 
prcasscssmcnt exercises (02 only); 

H3: Despite experimental classes (Xl) devoting up to 40% less class time to 
content knowledge acquisition than traditional classes (X2), no statisti- 
cally significant difference is expected between classes on content knowl- 
edge acquisition as measured by four objective tests; 

114: No significant difference is expected in instructor ratings, as measured by 
die Educational Testing Service’s (ETS) Student Instructional Rating 
(SIR), regardless of die mcdiod used (X 1 or X2). 
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The Literature 

Gordon and Howell (1959) identified the need for business schools to 
develop in students a set of basic managerial skills. Thirty years of study 
and applied experimentation have provided considerable knowledge 
about what these abilities arc (definitions and taxonomies) and about how 
to facilitate student learning of them (AACSB, 1987; Cameron & 
Whetten, 1983; Mintzberg, 1973; Porter & McKibben, 1988). In sum- 
mary, the literature suggests: 

1 . Although different labels arc used, for example, basic management skills, 
competencies, capabilities, skills and personal characteristics (SAPCs), or 
abilities, they appear to be describing the same or quite similar basic 
construct namely, what the students can actually do as a result of learning 
(AACSB, 1987; Accounting Education Change Commission, 1989; 
Boyalzis, 1982; Gordon & Howell, 1959). 

2. Similarly, much commonality exists between the variety of definitions and 
taxonomies that have been developed (Table 9.1). 

3. A variety of experiential methods have demonstrated effectiveness as means 
of facilitating student learning of these abilities in academic settings 
(Bigelow, 1988; Cameron & Whetten, 1983; Henderson, 1981; McEvoy. 
1989; see also Chapters 5 and 1 1 of this volume). 

4. Management faculty generally believe competency-based management edu- 
cation (CBME) is possible and desirable, further definition of competencies 
should be achieved, and the philosophical arguments used to oppose CBME 
arc logically inadequate and unsupported by theory and research (Albancsc, 
1989). 

5. The management skills development movement is stalled (Whetten, Windcs, 
May. & Bookstaver. Chapter 2 of this volume). Continued faculty resistance 
on philosophical, methodological, logistical, personal interests, and nega- 
tive sanctions, both institutional and professional, may explain this lack of 
progress. 

Methodology 

Random assignment of students to sections of the Principles of Man- 
agement course proved not to be feasible; thus, students were allowed to 
enroll freely in any of eight sections paired by time period (to control for 
class time period differences). To establish comparability of groups, 
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simple (one-way) analyses of variance tests were performed on the grade 
point averages of the students from the eight sections of Principles of 
Management (MGT 3310). No significant differences were found (F = 
1.65, p> .10), suggesting that llic groups were equivalent prior to the start 
of the experiment. 

Research Design 

The design was a special type of 2 x 2 factorial, called a Solomon Four 
Group. The factors were Pretest (Yes/No), and Instruction (Experimen- 
tal/Traditional). The design was chosen because it allows for the measure- 
ment of Pretest effects alone, Instruction effects with and without the 
effects of Pretest, and the assessment of the interaction between Pretest 
and Instruction. 

To control for differences attributable to teaching effectiveness, two 
instructors were chosen by the researchers and the department chair on 
the basis of equivalent experience, comparable student ratings, qualifica- 
tions, sex, and age. Each instructor was assigned to four equivalent 
sections of MGT 3310, representing each of the four design conditions. 

Measures 

Measures included (a) parallel assessment exercises (prcicsi-postiesi 
measures), (b) four traditional objective tests of content knowledge, and 
(c) student course evaluations. 

1. Assessment Exercises 

The pre- and postassessments of student management skills were 
equivalent sets of three assessment exercises developed by Development 
Dimensions International (DDI) to operationalize 14 skills and personal 
characteristics (SAPCs) defined by the Skills Diagnostic Program (SDP) 
of the Outcome Measurement Project (AACSB, 1987). Table 9.7 
describes the assessment exercises and die skill dimensions measured. 
DDI provided expert evaluation and scoring. Data on the reliability and 
validity of the pretest exercises arc reported in AACSB (1987). The 
parallel postasscssmcnt exercises were newly developed by DDI specif- 
ically for Uiis project, and no reliability or validity data were available at 
the time of die study. 
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The Literature 
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tive sanctions, both institutional and professional, may explain this lack of 
progress. 
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agement course proved not to be feasible; thus, students were allowed to 
enroll freely in any of eight sections paired by time period (to control for 
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simple (one-way) analyses of variance tests were performed on the grade 
point averages of the students from the eight sections of Principles of 
Management (MGT 3310). No significant differences were found (F = 
1 .65 , p > .10), suggesting that the groups were equivalent prior to the start 
of the experiment. 

Research Design 

The design was a special type of 2 x 2 factorial, called a Solomon Four 
Group. The factors were Pretest (Yes/No), and Instruction (Expcrimcn- 
tal/Traditional). The design was chosen because it allows for the measure- 
ment of Pretest effects alone, Instruction effects with and without the 
effects of Pretest, and the assessment of the interaction between Pretest 
and Instruction. 

To control for differences attributable to teaching effectiveness, two 
instructors were chosen by the researchers and the department chair on 
the basis of equivalent experience, comparable student ratings, qualifica- 
tions, sex, and age. Each instructor was assigned to four equivalent 
sections of MGT 3310, representing each of the four design conditions. 



Measures 

Measures included (a) parallel assessment exercises (pretest-posttest 
measures), (b) four traditional objective tests of content knowledge, and 
(c) student course evaluations. 

S 

1. Assessment Exercises 

The pre- and postassessments of student management skills were 
equivalent sets of three assessment exercises developed by Development 
Dimensions International (DDI) to operationalize 14 skills and personal 
characteristics (SAPCs) defined by the Skills Diagnostic Program (SDP) 
of the Outcome Measurement Project (AACSB, 1987). Tabic 9.7 
describes the assessment exercises and the skill dimensions measured. 
DDI provided expert evaluation and scoring. Data on the reliability and 
validity of the pretest exercises arc reported in AACSB (1987). The 
parallel postasscssmcnt exercises were newly developed by DDI specif- 
ically for this project, and no reliability or validity data were available at 
the time of die study. 



Ralph F. Mull in ei al. 



133 



(complex), interaction, in-basket (writing), job selection interview, and 
small group decision making. The asscssmcnt-ccntcr-typc exercises that 
were used involved students both in role play and m vicarious learning 
via assessment of teammates, using criteria specific assessment forms 
(Table 9.9). These were scheduled to approximately coincide with content 
knowledge coverage in the text. Thus, while traditional sections learned 
about a management principle or skill, the experimental sections learned 
both the principles and the applications. 

Traditional 

The traditional learning intervention (X2) covered the same textbook 
chapters and assignments and depended exclusively on the lecture-inten- 
sive method of teaching content knowledge. Both instructors were guided 
by how they had taught the course previously. 

Methods of Analysis 

To test hypotheses 1 (HI) and 2 (H2), a MANOVA analysis was 
performed on Uic pre- and posttreatment results (2 x 2 x 14 mixed design). 
The two bctwecn-subjccts factors were Treatment (Experimental vs. 
Traditional) and Experience (Post- Only vs. Pre- and Post-). To test 
hypothesis 3 (H3), ANOVAs were used to compare the difference in 
means between the experimental groups (XI and X2) and the traditional 
groups (X3 and X4) on student scores on the four content knowledge tests. 
ANOVA was also used to test hypothesis 4 (H4) to determine if any 
significant differences existed t>ct ween experimental and traditional treat- 
ments on student course evaluations. 

Results 

Hypothesis 1 (HI) was supported. The MANOVA indicated a signifi- 
cant (p < 0.001) learning effect due to the experimental treatment 
(Table 9.10). This provided statistically significant evidence that the 
experimental treatment, as compared to the traditional treatment, was 
effective in increasing overall student performance on the target set of 
basic management skills as measured by the pretest-posttest difference on 
DDEs measures. 

Hypothesis 2 (H2), which predicted that students taking the skills 
prcasscssmcnt would score higher on the postasscssmcnt, was not sup- 
ported. The MANOVA showed no significant difference (p = 0.534) 
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Table 9.9 

Description of Skills Learning Exercises 

1. Value s/Goal Setting 
Role: Self in career context 

Task: Garificaiion of personal values and goal-selling re. abilities 
Skill Dimensions: Judgment, Problem analysis. Impact (motivation) 

Time: 3 class periods (50 min.) and 3 hours out of class 

2. In-Basket Exercise 

Role: Manager, district government office 

Task: Evaluating and directing subordinates, project planning, etc. 

Skill Dimensions: Planning and organizing. Judgment, Problem analysis, Delegation, 
Control, Written communication 
Time: 3 class periods (50 min.) and 3 hours out of class 

3. Interview and Analysis 

Role: IIRD trainer, new employee, workcr/mcntor, or department 
Task: Interview and analyze new employee orientation program. 

Presentation: Written report of analysis and recommendations 

Skill Dimensions: Judgment, Analysis, Oral and Written communication 

Time: 3 class periods (50 nun.) and 1 1/2 hours out of class 

4. In-Basket Exercise 

Role: Middle-level manager, sales 

Task: Directing subordinates, dealing with customers and suppliers 
Presentation: Response in writing to the in-basket items 
Skill Dimensions: Judgment, Problem analysis. Written communication 
Time: 2 class periods (50 min.) and 1 hour out of class 

5. Selection Interview * 

Role: Personnel department, recruiter 

Task: Preparation and conduct of three selection interviews 

Skill Dimensions: Oral communication, analysis 

Time: 3 class periods (50 min.) and I 1/2 hours out of class 

6. Small Group Decision-Making 

Role: Sales manager (5 different roles, e.g., backgrounds) 

Task: Development of decision criteria and alternative solutions by group consensus 
(highly diverse member characteristics) 

Assessment: Day 1 -Group role play, Day 2-Observe group, Day 3-Discuss Skill 
Dimensions: Oral communication, Analysis, Group leadership 
Time: 3 class periods (50 min.) 



between the preasscssmcnt and postasscssmcnt sections and the 
postasscssmcni-only sections (Table 9.9). 
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Tabic 9.10 



Tests of Hypotheses HI and H2: SSPSX MANOVA 2 X 2 x 14 Mixed Design 


Source of Variation 


ss 


DF 


MS 


F 


Sig of F 


Within cells 


1909.42 


3484 


.55 






DIMENSIONS (14) 


1690.83 


13 


130.06 


237.32 


.000 


TRTMT BY DIM (HI) 


28.61 


13 


2.20 


4.02 


.000 


EXPER BY DIM (112) 


6.53 


13 


.50 


.92 


.534 


TRTMT BY EXPER BY DIM 


10.15 


13 


.78 


1.42 


.140 



N = 272 

MANOVA 2x14 

Treatment (Experimental vs. Traditional) 

with Dependent variables (14 Skill Dimensions) 

Canonical Cor. p-valuc 

interaction: -37 0.000 

Treatment (2) with Skill Dimensions (14) 



Hypothesis 3 (H3) was supported. No significant differences were 
found between the traditional and experimental sections on the four 
content knowledge exams (Table 9.1 1). 

Hypothesis 4 (H4) was likewise supported. No significant differences 
( p = 0.30) were found between the experimental and traditional sections 

on student course evaluations, measured by ETS’s SIR (Table 9. 12). 

* 

Discussion 

Support of hypothesis 1 (HI) provides additional evidence that selected 
basic management skills can be learned in an academic setting and dial 
the assessment center method is an effective pedagogy for undergraduate 
students. This finding confirms the results of the Phase I study. 

The lack of support of hypothesis 2 (H2) suggests that, without timely 
and developmental feedback, learning effects of pretest assessment exer- 
cises will not be significant. Considering the extent of concern given to 
control of pretest effects (sequence effects and interaction of testing and 
treatment) in research design (Campbell & Stanley, 1963), it is surprising 
that ill is highly reactive scvcn-and-onc-half hour pretest provided no 
statistically significant threat to internal validity. The results arc, however, 
perfectly consistent with and explained by social learning theory 
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Tabic 9.11 

Tcsl of Differences on Conlcnl Knowledge Exams — H3 



Design Test Group N = Mean F - p- 



One 


Experimental 


(100 pcs.) 


Traditional 


Two 


Experimental 


(100 pis.) 


Traditional 


Three 


Experimental 


(100 pis.) 


Traditional 


hour - Final 


Experimental 


(200 pis.) 


Traditional 


Combined 


Experimental 


(500 pis.) 


Traditional 



141 


83.5 






132 


85.5 


1.07 


0.71 


141 


82.2 






132 


83.2 


1.27 


0.16 


140 


80.7 






131 


82.3 


1.21 


0.28 


139 


159.3 






131 


159.3 


1.22 


0.24 


141 


402.8 






132 


408.5 


1.00 


0.996 



Tablc 9.12 

Tests of Differences in Student Course Evaluations 



Hypothesis: H4 
SAS ANOVA 

Question: How would you rate the quality of instruction in this course ? 



Source 


SS 


DF 


Mean S 


F Value 


PrF 


R’Square 


Model 


0.49 


, 1 


0.494 


1.08 


0.30 


0.005 


Error 


100.26 


219 


0.458 








Total 


100.75 


220 











Means: 1. Traditional (n = 11 1) 4.405 

2. Experimental (n = 110) 4.50 



Interpretation: No significant differences between experimental and traditional treatments on student 
instructional reports 



(Bandura, 1977). Since no feedback of results from the pretest was given 
to students, any immediate learning effects were not reinforced and, thus, 
not retained. 

Support of hypothesis 3 (H3) suggests that faculty concern regarding 
the deleterious effect dial teaching basic management skills might have 
on content knowledge acquisition is unfounded. The results were consis- 
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icnl with the intuitive proposition that a synergy exists in student learning 
of content knowledge through applications that simultaneously develop 
relevant basic skills. Outcomes were also consistent with evidence from 
20 years of experience with an ability-based learning program at Alvcmo 
College (Mentkowski & Doherty, 1984); namely, as students see the 
connection between content knowledge and “how they can use it out 
there” via skill applications, knowledge becomes more meaningful and 
important 

Support of hypothesis 4 (H4) suggests faculty concerns that students 
may react negatively to new methods, because they are different or 
because they place greater responsibility on the student for learning, arc 
probably unwarranted. 

Results of the current study clearly support the proposition that the 
assessment center method can be effective in teaching skill acquisition 
with no loss of content knowledge learning. However, the difference 
between experimental and traditional sections on posttest scores, while 
statistically significant, was of low magnitude (Canonical correlation = 
0.37). Additionally, when differences were tested for the 16 individual 
skill dimensions, only 6 were statistically significant, the differences were 
minor, and the results were mixed. The experimental treatment thus was 
effective compared to the traditional treatment, but students developed in 
small increments and not consistently. Artificial and weak effects of 
experimental manipulation are common. In this case, student grades were 
dependent only on completing the postasscssmcnt exercise and not on the 
quality of their performance. This may partially explain the low magni- 
tude of the experimental effects. Another explanation is that a single 
course can be expected to produce only minor gains in student skill 
development and then only for a limited number of management skills. 

Examination of the normed data reveals that, although students com- 
pared favorably to the AACSB/DDI Skills Diagnostic Program’s (SDP) 
norm group in Phase I of the study, when posttest scores from Phase II 
were compared to an external norm group of beginning and middle level 
managers, student skill levels were far below what employers may expect 
from graduates. These junior-level students may be expected to develop 
somewhat as they complete their program. However, since these skills arc 
not addressed by business school programs, gains may be expected to be 
minimal. Obviously, if performance levels of our students arc expected 
to measure up to those in the work force, more development is needed 
than can be expected in one course. The results, taken as a whole, thus 
strongly suggest the need for the adoption of a comprehensive abilitics- 
based model for the entire business school curriculum. 

0 u 
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Table 9.13 

Assessment-as-Leaming Model 

Assumptions About Learning 

Learning involves making an action out of knowledge — using knowledge to think, judge, 
decide, discover, interact, and create. 

An educator's best means of judging how well a learner has developed expected abilities 
is to look at corresponding behavior — thinking behavior, writing behavior, inquiry 
behavior, appreciating behavior, for instance. 

Learning increases development lly, even in its serendipitous aspects, when learners have 
a sense of what they are setting out to learn, a statement of explicit standards they must 
meet, and a way of seeing what they have learned. 

Essential Elements 

• Expected learning outcomes (including abilities) 

•Assessment as process involving multiple performances 

•Explicit criteria 

•Expert judgment 

•Productive feedback 

•Self-assessment 

Principles of Assessment 

1. Assessment is an integral part of learning. 

2. Assessment must involve a sample of behavior. 

3. Assessment must involve a performance of an ability representing the expected 
learning outcomes of a course, a program, a department, and/or the institution. 

4. Assessment involves expert judgment based on explicit criteria. 

5. Assessment must incorporate structured feedback. 

6. Assessment must occur in multiple modes and contexts. 

7. Assessment must incorporate an external dimension. 

8. Assessment is cumulative. 

9. Assessment instruments must incorporate open-ended possibilities for demonstrating 
a given ability. 

10. Self-assessment is an essential part of assessment as well as a goal of the process. 

It is an essential ability for the autonomous lifelong learner. 

SOURCE: Alvcmo College (1986) 



Development and evaluation of a new model for curricular and peda- 
gogical reform is the goal of the future phases of this study. Such a model 
will focus on die abililics-bascd general and program-specific student 
outcomes. These outcomes will be the organizing principles for redesign 
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of courses and methods widi extensive assessment of student perfor- 
mance, both within courses and comprehensive. The model’s assumptions 
about learning arc taken from Alvcmo College’s assessment as learning 
model (Table 9.13). 

The implications for future research include application of the model 
and method to the entire curriculum for a major, such as Management, 
and then generalizing the application to a variety of majors both within a 
college of business and in a variety of disciplines in other colleges of a 
comprehensive university. The final stage should be explicit definition of 
the entry-level criteria for abilities. These then can become the exit criteria 
for a university’s general education program. 

Conclusions 

The study suggests that renewed progress in Competency-Based Man- 
agement Education (CBME) may be achieved by business schools 
addressing the development of basic management skills systematically 
across the curriculum. This is a clear challenge made in Perspectives on 
Education (Accounting Education Change Commission, 1989). The 
required level of faculty and administrative planning, and matrix program 
coordination, is not evident in most business schools today. One logical 
response to this challenge is for groups like AACSB, the Academy of 
Management, the Organizational Behavior Teaching Conference and 
others to devote time at their conferences and meetings to focus on the 
combined issues of defining capabilities that can serve as curricular design 
objectives, systematic curricular reform, and new teaching and assess- 
ment methods. 
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MISSOURI STATE UNIVERSITY 



July 24, 1995 



Dr. Charles J. McClain 
Commissioner 

Coordinating Board for Higher Education 
3515 Amazonas Drive 
Jefferson City, MO 65109-5717 

Dear Dr. McClain: 

I am pleased to submit the enclosed Funding for Results (FFR) proposal for 
consideration by the Coordinating Board for Higher Education. As part of our FIPSE 
funded Continuous Process Improvement (CPI) project, which ended this past fall, some 20 
departments at Central have begun to use outcomes, assessment-as-leaming techniques, 
measures of input, diagnostic testing, and a variety of other methods to improve curricula, 
pedagogy, and student learning. These efforts to generate increasingly higher quality 
teaching and learning are proving to be effective. Goals and criteria of FFR and many 
current Central teaching and learning initiatives are highly complementary and can be 
mutually supportive. Clearly, FFR support is needed and can be used to expand Central’s 
CPI and other campus quality improvement efforts to more departments and academic 
support functions. 

For the first year, FY 1996, Central’s proposal is that its FFR efforts and program 
emphasis focus on the implementation of the recently approved University Studies 
program. The enclosed abstract describes in more detail the specific goals of the project 
and the responsibilities of our campus level FFR team. The team, chaired by Central’s 
Coordinator of Assessment and Testing Services, is truly campus-wide with members from 
all four colleges, the major administrative divisions, and the student body. Attachment A 
lists the FFR team members and the unit or group they represent. 

I strongly support this FFR initiative. Central Missouri State University has proven 
through its commitment of time, money, and effort, that it is serious about continuously 
improving student learning and development. University Studies is at the heart of all of 
our undergraduate programs and is the logical topic of focus for this pilot project. 



Should you have any questions or desire any additional information, do not hesitate to 
contact Mike Grelle, chair of the FFR team, at (816)543-4919 (Fax 543-8022). 



Sincerely, 




Ed Elliott 
President 
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CAMPUS LEVEL FFR PROJECT 



Background. 

Central’s University Studies (general education) Committee recommended, and the institution 
adopted in 1994, a revised University Studies Program. This program establishes as general 
education goals for all undergraduate students at Central, the four general outcomes of 
thinking, interacting, valuing, and communicating. According to the guidelines of the 
revised University Studies Program, all faculty teaching a university studies course must 
teach to and assess at least two of these four general outcomes. 

Although the University Studies document addresses the issues of assessment and evaluation, 
there are no directives indicating how the program is to be implemented. Central’s campus 
FFR team (See Attachment A for a list of team members.) therefore decided that the pilot 
project would focus on the implementation of the revised University Studies Program, which 
is set to begin in the summer of 1996. The FFR team established two major goals for the 
pilot project: 1) education of faculty in performance-based assessment related to thinking, 
communicating, interacting, and valuing, and 2) development of an entry-level 
comprehensive assessment of the general outcomes. Full implementation of the revised 
University Studies Program is a quality initiative that may require five to six years to 
complete. 

Learning and Teaching Issues. 

The primary goal of this project is to continuously improve and assess the quality of general 
education at Central Missouri State University. The project’s theme is a direct outgrowth of 
Central’s Mission and Goals Statement which reads in part, In fulfilling its responsibilities, 
the University ... provides an undergraduate liberal arts and science foundation with an 
emphasis on integrating critical thinking, communication skills, and technological 
applications into the curriculum across all disciplines." 

The addition of the four skill areas of thinking, communicating, interacting, and valuing to 
the University Studies Program creates a tremendous need for faculty development in 
performance-based assessment. Specifically, the faculty need to a) develop and integrate 
assessment-as-leaming teaching methods and strategies into their courses, and b) more 
effectively use feedback from in-class assessments to improve teaching and learning. 



Intervention Strategies. 



Since only a little more than a year remained between the adoption of the revised university 
studies program and its implementation, it was imperative that the faculty training portion of 
Phase 1 of the project began as soon as possible. Late last fall, four faculty Outcome 
Teams, based on the four general outcomes, were formed. These teams are the 
implementation arm of the University Studies Committee. The faculty team leaders directed 
monthly meetings for faculty (30-35) who were interested in developing expertise in 
performance-based assessment. This spring, a series of workshops were scheduled and 
planned by the Outcome Team leaders. On June 16-17, 1995, Central held a workshop for 
faculty who will teach university studies courses in the fall of 1996. The workshop was led 
by the team leaders and over 85 faculty attended. The workshop helped faculty develop new 
teaching and assessment strategies and skills that address the two outcomes they submitted in 
their approved course proposals. Additional training workshops are scheduled for 10/20/95, 
1/8-9/96, and 6/21-21/96 as well. By next fall, all faculty scheduled to teach in the revised 
university studies program should have received initial training in performance-based 
assessment. 

Data Elements and Measures. 

In order to have reasonably accurate estimates of the impact of the revised program on the 
quality of the general education of our students, the university needs to assess the 
competency of entering freshmen in the four outcome areas, as well as student reaction to the 
revised program as a whole. This fall a team of five faculty are to develop a comprehensive 
performance-based assessment of the four general outcomes that will be administered to the 
freshmen in the fall of 1996. This assessment will be used to establish a performance 
baseline for each student in the four outcome areas and to identify their relative strengths and 
weaknesses. The faculty team, along with other groups of faculty, employers, and 
professionals from outside the institution, will have the responsibility for developing test 
reliability and validity for this comprehensive assessment. Members of this external review 
team will also serve as evaluators of student performance. The comprehensive assessment 
should be ready for pilot testing in the spring of 1996 on a sample of graduating seniors. 
These seniors represent the last cohort to graduate from our institution under the old 
university studies program. These results will be used as a baseline against which students 
graduating in the future under the revised university studies program can be compared. 

Eventually, there will be three primary assessments of the four outcomes. The first 
assessment will be used to establish an entry-level baseline of students and to identify areas 
of strength and weakness. The second will be used to determine if a student is ready to enter 
the major program of his/her choice. The third assessment of the four general outcomes, in 
which the content area of the major will provide the context for the assessments, will be 
administered to seniors to determine if the student has met university general education 
(university studies) standards for graduation. This assessment regimen will also give us entry 
and exit performance measures, which may be used to estimate value added, and possibly 
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allow us to make comparisons against benchmark institutions. Attachment A presents a 
timeline for these and other projected assessments, as well as a brief description of each. 
These assessment data will become an important part of our permanent student data base. 

According to the guidelines of the revised University Studies Program, each faculty member 
teaching a university studies course will be asked to demonstrate how they used assessment 
information collected in their classes to improve their teaching. The mechanisms by which 
faculty will be audited to determine how they are using this feedback has not been established 
at this time. However, the Faculty Senate University Assessment Council has discussed 
using the program reviews (five or six of these occur campus wide annually) as a mechanism 
for such an audit. Another possible feedback mechanism is the annual report. Each 
academic department is currently required to submit an annual report which requests 
information on these same issues. The deans and provost review these reports and provide 
the chairs with a performance relation evaluation regarding their progress towards their stated 
goals. The University Studies Committee, which is also specifically charged with this 
responsibility, will submit a plan this coming year (1995-96) for how a systematic audit of 
University Studies courses could be conducted. 

Reward Structure. 

The University formally emphasizes excellence in teaching by requiring evidence of quality 
teaching as a prerequisite for all promotions. Central Missouri State University is currently 
considering the implementation of a common teaching evaluation instrument. One of the 
instruments being considered is the IDEA form. The IDEA form is a nationally-normed 
teacher evaluation instrument developed at Kansas State University, which can be used for 
both teacher evaluation (short form) or for improvement of teaching (long form). Central’s 
teacher evaluation program collects university-wide data on the quality of teaching and 
assessment. These data will be used to improve course content and student skills 
development within University Studies and individual disciplines, and to improve the faculty 
teaching practices. In addition, academic departments will utilize a variety of other teaching- 
effectiveness assessments to measure both pedagogy and curriculum effectiveness. 

The University is developing other incentives for faculty and faculty teams who provide 
leadership and positive changes in pedagogy and curriculum. The FFR team, along with 
several other campus groups, are examining plans for restructuring the current reward system 
to increase the focus of institutional resources on the core process of the university— student 
learning and development. The proposal to restructure the reward system, now in the 
developmental stage, will be initially implemented during the 1995-96 academic year. The 
envisioned reward system will provide monetary allocations to departments and units that 
provide evidence of improvement in student learning and teaching. 

In FY 1996, the Faculty Senate University Assessment Council, the University Studies 
Committee, the Academic Council, and the FFR team plan to use FFR dollars received by 
Central to support the university’s assessment plan and process. Initially, Central has set 
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aside approximately $30,000 from next year’s university assessment budget to fund 
department level and/or interdisciplinary assessment efforts dealing with the implementation 
of the revised University Studies Program. Other incentive funds, of a significant amount, 
will be allocated for assessment and faculty development. Central is developing a campus- 
wide agreement about the criteria and processes for distributing rewards based on achieving 
University Studies goals for academic year 1995-96 and beyond. The commitment of these 
funds demonstrates Central’s dedication to student learning and to this project. 

Project Evaluation. 

The project team, led by the University’s Assessment Coordinator, will be responsible for 
both formative and summative evaluation of the pilot project. There is considerable campus 
interest in measuring the specific impact of new pedagogical techniques on student learning 
of the four general outcomes (comprehensive performance assessments), on student 
perceptions of their learning and of the institution in general, and on student evaluation of 
teaching. These assessments will occur at both the classroom and university level and the 
information may be used primarily as feedback to faculty and students for the improvement 
of teaching and learning (formative evaluation). These data will contribute to a permanent 
data base which will allow faculty and others to track changes in student performance, 
attitudes, and teacher evaluations over time. The results from these comprehensive 
assessments will also be used for evaluation of the university studies program, and as 
possible evidence of quality performance (summative evaluation). 

SUMMARY. 

Central Missouri State University’s FFR pilot project represents the initial implementation 
phase of its revised University Studies program. Specifically, the pilot project will focus on 
1) the preparation of faculty to teach in the new university studies program, and 2) the 
development of comprehensive assessment instruments that measure student competency in 
the four outcome areas The project, an outgrowth of the university-wide CPI effort started 
at Central in 1991, is designed to improve teaching and learning, and to provide evidence of 
quality in the area of general education. 
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attachment a 



NAME 

Grelle, Michael 



CAMPUS LEVEL FFR COMMITTEE 

PHONE AREA 

4919 | Coordinator of Assessment 



Shaffer, Paul, ex officio 


4560 


Advisory Board for FFR 


Mees, J. P- 


4116 


Provost 



Hannah, Lyman 


Assessment Coordinator, Applied 
Science & Technology 


Karscig, Mark 


Assessment Coordinator, Business 
and Economics 


Mazza, Joe 


Assessment Coordinator, Arts and 
Sciences 


Mihalevich, Carol 


Chair, Faculty Senate University 
Assessment Council and Assessment 
Coordinator, Education and Human 
Services 


Carr, Kathy 


Outcomes Team Leader 


Sundberg, David 


Faculty Senate University 
Assessment Council 


Bennett, Marvin 


Finance and Administration 


Hicklin, Walt 


Student Affairs 


Hudson, Dee 


Admissions 


Nimmer, Don 


Institutional Research 


Schneider, Shannon 


Student Representative, Strategic 
Planning Council 


Student to be named . «■- 


Student Representative, Faculty 
Senate University Assessment 
Council 
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FACULTY QUESTIONNAIRE 



EXHIBIT -84 



PART I: Using the scale below, please rate the degree of your agreement or disagreement on each statement. Then, in 

the right-hand column, rate the degree of your agreement or disagreement with the statement: "My understanding of 
this statement has changed significantly since 1991." 



5_ Strongly Agree 
4_ Agree 

3_ Neither agree nor disagree 
_2_ Disagree 
J_ Strongly disagree 



Part I 





Present 


Change 

1991-94 


1 . 


Student learning is the primary purpose of an educational institution. 






2. 


Education goes beyond knowing to being able to do what one knows. 






3. 


Learning must be active and collaborative. 






4. 


Assessment is integral to learning. 






5. 


Abilities must be developed and assessed in multiple modes and contexts. 






6. 


Performance assessment-with explicit criteria, feedback and self assessment-is an effective strategy for 
ability-based, student-centered education. 






7. 


A coherent curriculum calls for faculty investment in a community of learning and Judgement. 






8. 


The process of implementation and institutionalization of a curriculum is as important as the curriculum: the 
process is dynamic, iterative, and continuous. 






9. 


Effective education requires faculty to be responsible for the process of defining outcomes and aligning all 
learning activities, teaching strategies, and assessment practices with these outcomes. 






10. 


Responsibility for education involves assessing student outcomes, documenting inputs, and relating student 
performance over time to the curriculum. 






11. 


Even if an institution has a sound curriculum in place, it should be changed. 






12. 


Course grades are not adequate measures of student learning and development. 






13. 


Assessing student competence at graduation is not the best means of assuring quality. 






14. 


The course credit completion model, our prevailing paradigm, is fundamentally flawed. 






15. 


1 am committed to do the hard work to continuously improve the learning process. 
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Continued 



Exhibit 8 a 



PART ll: Please rate the degree of your personal agreement or disagreement on each statement 

using the scale below. 



5_ Strongly Agree 
4_ Agree 

3_ Neither agree nor disagree 
2 _ Disagree 
1 Strongly disagree 
_0_ Not applicable 



Part II 



Between Fell 1991 end Spring 1994, faculty in my department (1) have .... 


Dept. 


Sell 


16. collaborated in development of a set of outcomes that define what our students should know and be able to do by 
qraduation. 






17. viewed the process of outcomes development valuable in terms of thinking more about how students learn and what 
colleagues are doing. 






18 thoroughly validated outcomes with constituents (students, graduates, experts, employers, graduate schools) 






19. revised curriculum to systematically assign responsibility for student learning outcomes, by level of development, to 
specific courses. 






20 developed criteria that define quality of performance for each outcome. 






21 accepted responsibility for developing student competence on specific outcomes integrated into course(s). 






22 changed our/my view of how and in what ways students learn. 






23. changed teaching behaviors. 






24. introduced more structured student learning applications/experiences into course(s). 






25. increased observation, assessment, and feedback of student performances. 






increased development and use of assessment forms. 






27 involved students in self-assessment of their performance. 






28 involved students in peer assessment of their performance. 






29. increased use of students teams in learning. 






30 increased use of assessable 'products' of learning e.g., papers, projects, portfolios. 






31 specifically increased use of writing and written assignments. 






32 specifically, increased use of oral-visual presentation 






33. specifically, increased use of exercises that require higher thinking skills level. 







Part III: Lessons Learned. Please describe what you would tell a faculty friend whose university 
was beginning a major project to improve the student learning process. What have you learned 
that is most important? What have you learned that should be avoided? (Continue comments on 
the back of this sheet if necessary.) 
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STUDENT QUESTIONNAIRE 



Exhibit 8b 



Please rate the degree of your personal agreement or disagreement on each statement using the 
scale below. 



5_ Strongly Agree 
4_ Agree 

Neither agree nor disagree 
2 _ Disagree 
J_ Strongly disagree 
0_ Not applicable 



Faculty in my major area have .... 


Rating 


1. developed of a set of outcomes that define what students in my major should know and 
be able to do by graduation (major outcomes). 




2. explained these major outcomes so that 1 understand what is expected of me by 
graduation 




3. involved other constituents (students, graduates, experts, employers, graduate schools) in 
developing these outcomes. 




4. systematically assigned responsibility for student learning outcomes, by level of 
development, to specific courses. 




5. developed criteria that define quality of performance for each outcome. 




6. developed specific outcomes for each course. 




7. helped change my view of how and in what ways students learn. 




8. used student learning applications/experiences in course(s). 




9. observed and assessed student performances and feedback. 




10. developed and used assessment forms based explicit performance criteria. 




11. involved students in self-assessment of their own performance. 




12. Involved students in peer assessment of other student performances. 




13. used student teams in learning. 




14. used assessable "products" of learning e.g., papers, projects, presentations, portfolios. 




15. specifically, used writing and written assignments. 




16. specifically, used oral-visual presentation assignments 




17. specifically, used exercises that require higher thinking skill levels. 





18. I am a (circle one): Freshman Sophomore Junior Senior 

1 9. My major is 
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Central's Assessment Data System 
Outline of CP] Measurement Goals and Plan 



Outline of Measurement Areas: 

The outline of CPI Measurement Goals and Plan attempts to address the latter of the 
two concerns expressed above. An informal group of faculty met to identify those 
general classes (categories) of measurement areas which are integral to the CPI model 
and its intended impact on student learning (see Data Outline & Flowchart). To 
evaluate the effectiveness of individual faculty members efforts and of the process as a 
whole, it was decided that there should be 5 classes of measures to capture all relevant 
data sources (date at which majority of area's instruments could be implemented in 
parentheses) : 

• Customer Satisfaction or Perception Measures (In-place/1995) 

• Learning Process Measures (1994) 

• Measures of Student Performance (Currently in development/ 1994) 

• Outcome Measures (1995-96) 

• Other "Outside" and/or Demographic Measures (In-Place) 

The outline addresses each of these areas and identifies specific examples and 
instruments which would/could be used to obtain relevant information. 

D. Nimmer and M. Karscig have continued to meet this semester to examine what 
resources and mechanisms would need to be put into place to begin the development 
of the data bases which would aid in the implementation of the above instruments. 
Much thought discussion have been given to the data which are currently available and 
give insighT into student performance, demographics, customer satisfaction and the 
learning process as a whole. Some of the short term, preliminary goals for the 
designing of an assessment data system here at Central would include (again, proposed 
completion dates in parentheses): 

• Development of matrix which would identify what is to be measuied, when, 
where, and how they were to be measured. (End of Spring 1994) 

• Where the individual data elements will be housed - department, college, 
university levels. Where would the data be entered and who would have access 
to them. (Summer, 1994) 

• When and where the data can be purged over time or with changes in 
courses/curricula and/or of a students major. (Summer, 1994) 

These issues must be addressed first before the actual work on the development of the 
assessment data system (ADS) and the planning for the hardware/programming 
resources necessary to sustain it are initiated. 
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II. 



III. 



IV. 

V. 




CPI Measurement Goals and Plan - DRAFT 
]upa<sI IREMENT AREAS 

CUSTOMER SATISFACTION (PERCEPTION) MEASURES: 

A. External Customers 

1. Employer surveys 

2. Graduate survey (s) 

a. Graduating Seniors 

b. Cohort groups .longitudinal data 

3 Graduate/Professional Schools 

B. Internal Customers 

1. Student Satisfaction 

a. IDEA System (Course Evaluations) 

b. Student Defined Benefits (re. Brower article) 

2. Faculty 

a. Organizational Culture/environnient (instrument) 

b. Faculty supplier-customer relationships (QFD) 

c. Faculty satisfaction with CPI process 

j Contact/relationships with external customers and associations (State of the art in the 

discipline) 

LEARNING PROCESS MEASURES: 

A. Outcomes: 

B. Curricular 

C. Instructional 

D. Student Learning Process 

• Pre-admission - ACT 

• Admission/Financial Aid 

• Advisement 

• Placement 

I Student evaluation & feedback on learning (e.g. 1 min. paper, focus groups, journals, etc) 

2. Student involvement in process 

• Inside class (survey frequency & depth of involvement) 

• External (cross functional teams on curriculum, outcomes, assessment), etc. 

3. Co-curricular 
STUDENT PERFORMANCE: 

A. External (Comprehcnsivc-across classes): 

L Where Assessed: 

•Entry to the University Studies (Placement, ACT) * Entry into Major Assessment 
•Mid-point Assessment of Intellectual skills (US) ’Exit from Major Assessment 
•Exit University Studies 
2. Types of Assessments: 

•Objective tests ’Oral presentations 

•Written reports •Portfolios 

•In-Basket exercises 

B. Internal (within courses) 

•Objective Tests on Content Knowledge 
•Essay Questions 

•Writing Projects (Letters, Reports, Strategic Analyses, Employee Manuals, etc.) 

• Oral Presentations (self evaluation, peer evaluation, instructor evaluation, and videotaping) 

• Small-Group Projects (self evaluation, peer evaluation, instructor evaluation, and videotaping) 

• Instructor rating of listening skills in lecture situation 

• Instructor evaluation of role-playing situations 

C. Evaluation of Assessment design and measures 
OUTCOME MEASURES (Program): 

OTHER MEASURES 

A. Student Demographics 

B. CBHE ’’quality” variables 

a. Number of African- American students graduating 

b. Number of students taking standardized tests 

c. Number of students taking local tests 

d. Number of students in Critical Disciplines 

e. ? 

C. Support Services: 
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Curriculum Planning and Student Learning 
CPI with Student and Program Feedback Loops 
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rom its very 
beginning as 
State Normal 
School Number Two, Central 
Missouri State University has put 
the people of Missouri first in its 
service, planning and develop- 
ment. Central is committed to the 
citizens of Missouri, and works 
closely in partnership with the 
people of this great state to carry 
out its mission. The education of students is at the 
forefront of all the institution's initiatives. Central is 
unquestionably, "The People's University." 

Those of us in public education have an ethical 
and moral obligation to take the very broad base 
of society's human talents and abilities and identify 
how those can be enhanced through the 
educational process. At Central, we are dedicated 
to doing what we can to improve the lifestyles and 
career opportunities of all the state's citizens. We 
work to ensure the education of not only Missouri's 
children but also of the adults who want to continue 
their schooling and of the multitude of workers who 
lose or change their jobs each year and need 
retraining. 

At Central, we have stopped segmenting 
education and have accepted it as a seamless, 
interwoven, lifelong process. The practice by some 
colleges and universities of controlling quality by 
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President Ed Elliott 



restricting access is short-sighted. 
Limiting admission to only 
top-ranked students does not 
ensure the quality of an institution 
nor the value of an education. At 
Central, we don't spend our time 
devising ways to restrict access. 
Instead, we dedicate our efforts to 
improving the educational process 
and design, as we focus on 
quality graduates and 

performance standards achieved by our students. 

Central offers many programs not found at 
other Missouri colleges and universities, and many 
of our programs are nationally or internationally 
known. We are one of the leaders in discipline- 
specific accreditations within Missouri higher 
education, and other colleges and universities have 
used some of our high-caliber programs as models 
at their institutions. 

Through extensive collaborative efforts, we 
have developed and put into place an innovative 
and systematic teaching/learning/assessment 
model. We believe this approach, Continuous 
Process Improvement (CPI), will have an impact on 
the future of higher education in the state of 
Missouri and, possibly, all of education. As an 
extension of our CPI program, Central is leading a 
collaborative effort with elementary and secondary 
education to design, develop and test new 
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pre-kindergarten through 
baccalaureate degree learning 
process models. These partnerships 
between schools, students, parents, 
government, business and the private 
sector are contributing to quality 
education and to the successful 
reformation of Missouri's education system. 

One of Central's key strengths lies at the heart 
of institutional integrity — a willingness to openly 
discuss and clarify the university's mission. We 
have established clear, measurable and 
achievable goals and objectives as a function of 
that mission and have created plans of action to 
accomplish agreed-upon educational outcomes. 
Maintaining high standards and achieving the 
designated mission are our highest priorities. 

It is also important that we plan for academic 
and administrative changes to better achieve our 
mission and maintain standards. Task forces are 
currently at work on our campus to examine the 
university's organizational structure and admission 
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policies, as well as resource 
reallocation and better ways to 
make campus-community 
connections throughout our service 
area. Through these task forces we 
are exploring new ways to better 
serve the people of Missouri. 

Public education truly belongs to everyone, at 
all times. The people are the academy, and the 
process of learning is one that should fully 
empower every citizen. The strength of Missouri's 
education system will direct the state's future; and 
Central Missouri State University is dedicated to 
providing the people of our state with the highest 
quality educational system and experiences 
possible. 

Please take some time to read this report. It is 
by no means all-inclusive, but it does provide some 
examples from the past year of the many ways 
Central Missouri State University is meeting its 
responsibilities as The People's University. 

5 £ ^ 

5 President 
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CENTRAL'S CPI RETREAT: 1994 
"Delivering the Quality Promise" 
EVALUATION 




Continuous assessment/measurement is a necessary condidtion for continuous improvement. 
Please help us improve by providing us with your most thoughtful feedback on Central's CPI 
Retreat 1994. 

PART I: Using the scale below, please rate the degree of your agreement or disagreement on 

each statement. 



5 Strongly Agree 
4 Agree 

3 Neither agree nor disagree 
2 Disagree 
1 Strongly disagree 

Part I 





Rating 


1. 


1 have a clearer understanding of what Central's CPI is and is not. 




2. 


1 have a much clearer understanding of why CPI is so important to Central's future. 




3. 


The 24 goals (1994-97) are clearly stated. 




4. 


1 believe the 24 goals are appropriate and focused on priorities. 




5. 


1 have a personal responsibility to “Deliver the Quality Promise. - 




6. 


Ed Elliott's vision for Central and where we are headed in the next few years is clear. 




7. 


How we can collaborate to “Deliver the Quality Promise* is clearer than before the retreat. 




8. 


1 received the notebook in advance of the retreat. 




9. 


The four priority objectives for the retreat were made explicit and clear. 




10. 


1 agree that we all have much to learn and that a major increase in faculty and staff participation in CPI/Guality/skill 
training is a top priority. 




11. 


1 will personally lead in deciding on planning and scheduling training. 




12. 


The four general outcomes provide sound criteria for coordination of university and major studies. 




13. 


The four outcome task teams can provide the basis for faculty learning and comprehensive assessment. 




14. 


1 will personally participate in one of the four teams. 




15. 


1 will persuade other faculty to participate in an outcomes task team. 




16. 


We must spend much more time and money on CPI/Quality training. 




17. 


If necessary to assure attendance, the academic calendar should be changed. 




18. 


1 now understand why we must work with K-12 schools and teachers on Quality improvement. 




19. 


The breakout session 1 attended was very helpful. 




20. 


The goals for the breakout session were made explicit and clear. 




21. 


My overall rating for the CPI Retreat: 1994 is excellent. 
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You can count on me to be a Central CPI leader. 
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PART II: Comments. 

i. 

23. Of the 24 CPI goals, I believe the following are most important: 



24. The most important types of training needed are (please rank or rate, see Goal# 2): 



25. Who should be responsible for planning training? 



26. Who should be responsible for training scheduling and attendance? 



27. How can we best obtain commitment from those in your department to attend the necessary training? 



Breakout session attended: A (Gen Ed/Outcomes) , B (Data) , C (PK-16) 

28. What is the most important thing you learned in this breakout session? 



29. What questions still remain or what material was ambiguous? 



30. Which of the goals of the breakout session were most clearly achieved for you? 
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CENTRAL’S CPI RETREAT: 1994 
"Delivering the Quality Promise" 

Saturday, July 16, 1994 
REPORT TO THE PROVOST 

Due: August 18, 1994 

College: 

Attendance: ' 

1. Action Plan to implement Goal# 6: To have the principles and components of the CPI 
model operationalized in all departments and graduate programs with a minimum of 
85% faculty involvement and participation by September 1997. Refer to Tab 5, 

College, "DRAFT DRAFT DRAFT," the ten major components of the CPI model. 

Department Component Goals (each department! Responsibility Timeline 



2. General Outcomes Team: 

Goal# 8: To operationalize assessment/training teams representing faculty members 
from all colleges organized on the basis of the general outcomes-communication, 
human interaction, critical thinking, and valuing. 

Department Communication Interaction Thinking Valuing 
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3. ' Training 

Goal# 2: To design, develop, and implement a comprehensive training program for 
faculty (450), staff (200), and administrators (20). 

Department Type of Training Number and names of Faculty Responsibility 

Dates 



Types of Training: 

1. Basic CPI Model, TQM Principles and Tools, Assessment-as-Learning, etc. 

2. Applied Learning: Design and development of student learning experiences, criteria, 
assessment means, and feedback methods for a course or learning module 

3. Quality leadership: Process improvement defined, analyzed, and designed by a 
cross-functional team. 

4. Special Skill Training: Team learning, time management, management of teams and 
meetings, measurement, and classroom use of quality tools. 

5. Training of facilitators: 

6. Interorganizational Faculty training with PK-12 Teachers and administrators. 

7. Other: 



POTENTIAL TRAINING DATES: 

1. Thursday/Friday Prior to Plan Days 

2. Friday (Mid-Fall) - District Teachers Mtg. 

3. End of semester (Fall) after finals 

4. End of Spring semester 

5. Friday in June 

6. Summer Planning Retreat 

TOTAL DAYS 



August 11/2 days 
October 1 day 
December 11/2 days 
May 11/2 days 
June 1 day 
July 11/2 days 
8 days 



COORDINATING GROUP: Provost, CPI-Director, Assessment Director 
Deans, Chairs, Cabinet, Directors 
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Exhibit 12 

Dissemination Example 
Ralph Mullin 
Project Director 

1. Books Chapters 

Mullin, R., Shaffer, P., and Grelle, M., "A Study of the Assessment Center 
Method of Teaching Basic Management Skills," Chapter 9, Management 
Skills . John Bigelow, ed., 1991. 

Mullin, Ralph F., Wilson George, and Grelle, Michael, "CQI: A Model for 
Reform Based on Assessment as Learning and TQM." Continuous Quality 
Improvement: Making the Transition to Education . Dean Hubbard (ed.), 
Prescott Publishing, Maryville, MO 1993 

Mullin, Ralph F., Wilson George, and Grelle, Michael, "Decade of 
Transformation at Central Missouri State University," Continuous Quality 
Improvement: Making the Transition to Education . 2nd Edition, Dean Hubbard 
(ed.), Prescott Publishing, Maryville, MO 1993 

Mullin, Ralph F., 1994, "Central Missouri State University: TQM Pilot 
Experience." Excellence in Missouri Foundation: Quality in Education . John 
Politi, Editor, Jefferson City, MO, December 1994. 

2. Other Publications. 

Mullin, Ralph F., "Changing the Teaching-Learning Paradigm: Central Missouri 
State University." Freshman Year Experience Newsletter . Vol.4, No. 4, Spring 
1992. 

Mullin, Ralph F., "Quality in Higher Education," Kansas City Business Journal , 
September, 1994. 

3. Articles. 

Mullin, Ralph F. (1995) "Quality in Education: Central Missouri 

State's Decade of Transformation." Missouri Quality Award: Pilot Studies in 
Quality Education . John Politi, Editor. Jefferson City, MO: Excellence in 
Missouri Foundation, March 

Mullin, Ralph F. and Wilson George, "Quality in Education: A New 

Paradigm and Exemplar." Continuous Quality Improvement: Making the 
Transition to Education . 2nd Edition. Dean Hubbard (ed.) Prescott Publishing, 
Maryville, MO 1995. 



Mullin, Ralph F., Wilson George, and Grelle, Michael J. (1993) 

"TQM: Revolution or Just Another Fad?" Commissioned Paper for the 
American Association for Higher Education "Double Feature Conference" on 
Assessment and Continuous Quality Improvement, Chicago, IL: June 9-12 

Mullin, Ralph F., Wilson George, and Grelle, Michael J. (1993). 

"CQI: A Model for Reform Based on Assessment as Learning and TQM." 
Continuous Quality Improvement: Making the Transition to Education . Dean 
Hubbard (ed.) Maryville, MO: Prescott Publishing. 

Mullin, Ralph F. and Grelle, Michael J. (1992). "Changing the 

Teaching-Learning Paradigm: Central Missouri State University." Freshman 
Year Experience Newsletter . Spring Vol.4,No.4 (30,000 Circulation) 

Mullin, Ralph F. and Grelle, Michael J. (1993) FIPSE Project 

Directors' Annual Program Book . Washington, DC: U.S. Department of 
Education, October. 

Mullin, Ralph F. and Grelle, Michael J. (1992). "Basic Reform of 

Teaching and Curriculum: An Outcomes Driven Contiuous Improvement (CPI) 
Model." Proceedings Freshman Year Experience . Maintaining the Momentum, 
Teaching Meeting, Kansas City, MO, April 2-4. 

Mullin, Ralph F. and Grelle, Michael J. (1992). FIPSE Project 

Directors' Annual Program Book . Washington, DC: U.S. Department of 
Education, October. 

Mullin, Ralph F., Shaffer, Paul L. and Grelle, Michael J. (1991). 

"A Study of the Assessment Center Method of Teaching Basic Management 
Skills." Chapter 9, Managerial Skills: Explorations in Practical Knowledge (ed.) 
John D. Bigelow. Newbury Park: Sage Publications. 

Mullin, Ralph F. and Grelle, Michael J. (1991) FIPSE Project 

Directors' Annual Program Book . Washington, DC: U.S. Department of 
Education, November. 

B. Papers/Presentations 

August 1-4, 1995. "Transforming Faculty Work, Our Learning 

System and the University for the 21st Century: Physician Heal Thyself." 
Academy of Management Conference. Vancouver, B. C., Canada 

May 1-4, 1995. Transforming Education: The Assessment Center 
Method as a Means." International Congress on the 
Assessment Center Method. Kansas City, MO 







March 3 1 -April 1, 1995. Midwest Academy of Management. 

St. Louis, MO 

February 6-8, 1995. Malcolm Baldrige National Quality Award. 

"The Quest for Excellence VII." Washington, D.C. 

February 1995. Invited consultant for review on assessment. 

Samford University, Birmingham, Alabama. 

January 12, 1994. One-Day Seminar for Daniel Seymour, TQM in 
higher education author and consultant, Palm Springs, CA 

February 11, 1994. "A New Approach to Learning: The Continuous 

Process Improvement Model for Undergraduate Education." Invited workshop 
(20) by College of Agriculture, University of Missouri, Columbia, MO 

February 18, 1994. "A Model of Seamless PK-16 Education," 

Interdistrict Educational Alliance (15 Superintendants), Learning Exchange, 
Kansas City, MO. In conjunction with a planning meeting with Connie 
Campbell, Learning Exchange staff and CMSU College of Education leaders. 

February 23, 1994. Central District, Missouri Association of 

School Adminstrators, "A Model for Reform to Develop a Seamless PK-12 
System." School District Superintendants (43), Central Missouri State 
University, Warrensburg, MO 

March 1-2, 1994. "Forget the Silly Notion That TQM Will 

Transform Education." First Annual Quality Conference, Ozarks Quality 
Council, AQP, Springfield, MO 

March 4, 1994. Requested workshop on CPI for visiting faculty 

of East Central Oklahoma State University (Bill Osborne and Ed Russell) at 
Central Missouri State University. 

March 11, 1994. Luncheon with potential funding partners for 

Phase V (1994-97). John Laney (Hall Family Foundations); Tom Sprott, 
Trading Area General Manager, IBM; Bill Berkeley, Tension Envelope, COB, 
Learning Exchange, Kansas City, MO 

May 5-7, 1994. Invited Panelist "Outcomes Assessment." 

Western Regional American Accounting Association, 

Portland, Oregon tie to AACSB Airfare paid $300 Curt Deberg (FIPSE), FAX 
916/898-4584, T- 6463 1) Decade 2) AAHE 3) Bio Mary Alice Seville T: 
503/737-6060; FAX 503/737-4890 



June 2, 1994. Luncheon with Ed Elliott, Connie Campbell and 

potential funding partners for Phase V (1994-97). Marilyn Kourilsky and Gary 
Baker, Kauffman Foundation; Regional for U. S. Department of 
Education; Kansas City Community Foundation and Affiliated Trusts; 

UMKC at the Learning Exchange, Kansas City, MO 

June 1 1, 1994. "A Model of Seamless PK-16 Education," One-day 
Conference at Central with representatives of 

June 23-25, 1994. "Learning Managerial Skills Across The 

Curriculum: What's the Next Step?" with John D. Bigelow, Stephen 
Robbins, and David A. Whetten. International Organizational Behavior 
Teaching Conference, Windsor, Ontario, Canada. 

September 27-29, 1994. "Creating a Quality Learning System: 

Faculty Development." Two presentations with Dean Shaffer, 

Craig Christie and Lyman Porter at the AACSB Continuous Improvement 
Symposium, St. Louis, MO: American Assembly of Collegiate Schools of 
Business. 

October 21-24, 1994. "Assessment of Educational Outcomes." 

Series of meetings with College Teaching Committee, chairs and faculty of 
the College of Commerce, DePaul University, Chicago, IL. Invited consultant. 

October 28, 1994. "Quality, Assessment and Learning: Using 

Profound Knowledge for Quantum Improvement of the Student Learning 
Proccess." Metropolitan Community Colleges, In-Service Professional 
Development, Kansas City, MO 

November 3-5, 1 994. "Transforming Education: Start With 

Faculty, Bust The Paradigm, and Focus on Student Learning." Transforming 
Eduacation with Quality: Blueprints for Success. Fifth Annual Quality in 
Education Symposium. Salt Lake City, Utah. Two presentations 

November 17-20, 1994. "The Seamless PK-16 System: Superceding 
the Carnegie Unit and Course-Credit Models." AAHE Fifth National 
Conference on School/College Collaboration. Washington D.C. 

December 7, 1994. "Applying Quality and Assessment-as-Learning 

Principles to Improve Student Learning." Workshop sponsored by the Kansas 
City Professional Development Council, Longview Community College. 

December 14, 1993. Seminar on CPI for visiting faculty (5), The 
University of Missouri, Columbia, MO 



November 15-18, 1993. Three-Day Seminar for Sue Hodges Moore, 

University of Louisville, Louisville, TN. TQM in strategic planning and learning 
improvement in higher education. 

October 27-28, 1993. Missouri Quality Conference, Jefferson 

City, MO ’’Transforming Education K-16: Quit Tinkering and Change the 
System.” 

October 18, 1993. One-Day Seminar for Peter Bishop, University 
of Houston, Clearlake, TX (IBM Award Grant Institution) 

October 21, 1993. Invited Seminar on CPI for Deans Jefferson 

and Lawrence, Acceditation Team, American Assembly of Collegiant Schools 
of Business (AACSB), D208, Central Missouri State University. 

October 8-10, 1993. FIPSE Project Directors' Meeting, 

Washington, D.C., "Practical Steps Toward Continuous Process Improvement 
(CPI): An Interactive Workshop" with Joan Straumanis and Mike Grelle 

August 11, 1993. Board of Regents, Central Missouri State 
University, CPI Report and plans. 

July 27-30, 1993. "TQM: Transforming the Core Learning 

Process, K-16." Fourth Annual Total Quality Symposium, Continuous Quality 
Improvement. Kansas City, MO. Presentation repeated Wednesday and 
Thursday 

July 22-23, 1993. First Annual University Academic Planning 

Retreat on CPI. Planning, preparation, and coordination of this meeting to 
involve deans, chairs, assessment coordinators, and selected staff and 
administrators (70). 

July 18-21, 1993. Society for College and University Planning, 

Boston, MA. "Planning for Contiuous Improvement: Start with the Faculty, 
Bust the Paradigm, and Focus on Student Learning." 

July 14-15, 1993. Retreat for conceptualizing re-engineering 

with President Elliott and Quality Advisory Team, Ritz Carlton Hotel, Kansas 
City, MO 

June 9-12, 1993. American Association for Higher Education 

"Double Feature Conference" on Assessment and Continuous Quality 
Improvement, Chicago, IL. (1) Criticism and Discussion of Commissioned 
Paper 



June 9-12, 1993. American Association for Higher Education 

"Double Feature Conference” on Assessment and Continuous Quality 
Improvement, Chicago, IL. "Faculty Leadership of TQM: A Practical Model 
for Change," In-Conference Workshop. 

May 19, 1993. Board of Regents, Central Missouri State 
University, CPI Report and plans. 

May 14, 1993. Workshop for faculty of North Central Community 

College, Trenton, MO at Central Missouri State University on CPI and reform 
of student learning process. 

April 28-30, 1993. "Improving Teaching Effectiveness: A Total 

Quality Systems Approach Starting with Faculty, Busting The Paradigm, and 
Focusing on Student Learning," with Rodney Sherman. Fourth Annual 
Business Teaching Conference, Minneapolis, MN 

April 26, 1993. Invited Workshops (2) with Northwwest Missouri 

State University Faculty, Chairs and Administrators; plus Luncheon with 
President Dean Hubbard and Cabinet on CPI. Maryville, MO 

March 17, 1993. Invited Seminar on CPI for Dean Jerry Johnson 

University of South Dakota, Acceditation Team, American Assembly of 
Collegiant Schools of Business (AACSB), D208, Central Missouri State 
University. 

March 12, 1993. Kansas City Regional Consortium on Higher 

Education (KIRCHE) Workshop, Avila College, Kansas City, MO. Two 
presentations on Central's assessment experience. 

March 5, 1993. Invited Seminar on CPI for Dean Quiester 

Craig, President, American Assembly of Collegiant Schools of Business 
(AACSB), D208, Central Missouri State University. 

February 19, 1993. Missouri Board of Education and Department 

of Elementary and Secondary Education (DESE), Tan-Tar-A Lodge, Osage 
City, MO 

February 16, 1993. President's Cabinet, Central Missouri State 
University. CPI Update Presentation and discussion 

February 5, 1993. Commissioner of Education (DESE), Robert 

Bartman, Jefferson City, MO. Presentation on Central's CPI project with Joe 
Huber, Dean, College of Education & Human Services and Ted Garten, Chair, 
Curriculum and Instruction Department. This resulted in agreement to work 
together on reform and our presentation to the Board 2/19/93 



January 26, 1993. Open Forum on CPI with students (@100) with 
George Wilson and Mike Grelle, University Union Ballroom 

January 20, 1993. Open Forum on CPI with students (@100) with 
George Wilson and Mike Grelle, University Union Ballroom 

January 13, 1992. "Update on CPI/FIPSE Project" President's 
Cabinet, Central Missouri State University. 

April 2-4, 1992. "Basic Reform of Teaching and Curriculum: 

An Outcomes Driven Contiuous Improvement (CPI) Model." Maintaining the 
Momentum, Teaching Meeting, Kansas City, MO 

June 20-23, 1992. "TQM for Faculty: A Practical Model for 

Improving Student Learning." 7th Annual American Association for Higher 
Education Assessment Conference, Miami Beach, FL 

August 10, 1992. "CPI/FIPSE Update and Discussion" 

President's Cabinet and Academic Deans, Central Missouri State University. 

August 12, 1992. "CPI/FIPSE Update and Discussion" 

Board of Regents, Central Missouri State University. 

October 16-17, 1992. "Total Quality Management and Systemic 

Reform of Higher Education: An Outcomes Driven Continuous Process 
Improvement (CPI) Model." This workshop was repeated on the second day 
to assure complete coverage. FIPSE Project Directors' Meeting, U.S. 
Department of Education, Washington, D.C. 

December 7-8, 1991. Two-day workshop on assessment design. 

James Roth, Professor, History, Alverno College (consultant) Central Missouri 
State University, CPI/FIPSE Project. Planning and coordinating. 

November 1-4, 1991, FIPSE Project Directors' Meeting, U.S. 

Department of Education, Washington, D.C. 

June 27, 1991, (1) Connections: Learning Principles of Human 

Interaction and Management from Art, Literature and Movies (with Larry 
Michaelsen); and (2) How Can Basic Management Skills be best Learned (with 
John Bigelow and Mel McKnight). International Organizational Behavior 
Teaching Conference, Bellingham, WA. 

May 8, 1991, Teaching Basic Management Skills. 19th 

International Conference on the Assessment Center Method, Toronto, Canada 
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April 2-4, 1991, "Basic Reform of Teaching and Curriculum: An 

Outcomes Driven Continuous Process Improvement (CPI) Model." The 
Freshman Experience, Kansas City, MO 

April 21, 1990, Assessment and the Assessment Center Method as 
Means of Improving Student Learning in Business Schools: 

Alverno College's Performance-Based, Outcome-Oriented 
Curriculum. Thirty-third Annual Meeting of the Midwest 
Division of the Academy of Management, Milwaukee, Wisconsin. 

June 27, 1991, (1) Connections: Learning Principles of Human 

Interaction and Management from Art, Literature and Movies (with Larry 
Michaelsen); and (2) How Can Basic Management Skills be best Learned (with 
John Bigelow and Mel McKnight). International Organizational Behavior 
Teaching Conference, Bellingham, WA. 

May 8, 1991, Teaching Basic Management Skills. 19th 

International Conference on the Assessment Center Method, Toronto, Canada 

April 21, 1990, Assessment and the Assessment Center Method as 

Means of Improving Student Learning in Business Schools: Alverno College's 
Performance-Based, Outcome-Oriented Curriculum. Thirty-third Annual 
Meeting of the Midwest Division of the Academy of Management, Milwaukee, 
Wisconsin. 
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What Authorities Have to Say About Central’s CPI Quality Reform 

"I am enormously impressed with what CMSU has already put into place so far in developing measures 
and assessing student learning and performance outcomes in their first four phases, and, frankly, I am in 
awe at what they propose to do in the next phase (Phase V). If they can actually deliver on what they 
promise - and I am convinced that they can - then it will be a signal accomplishment with exceedingly 
important implications for higher education. . . If we are ever to have a substantial test of both the 
feasibility and the degree of success of a competency-based assessment approach to higher education, this 
should be it." 

Lyman W. Porter, Professor, University of California, Irvine 
Past Board of Directors and Futures Project Chair, AACSB 

"In my experience, I can name a handful of institutions that are "thinking and doing" what other 
institutions are talking about — that is, recreating themselves. Central Missouri State University is one of 
those institutions. . . They are not tinkering at the edges. They are looking to establish fundamental 
systemic change. They are documenting their work and writing a history of reform as they work to 
establish change. They are advocates. I have heard Ralph Mullin and others at Central speak at national 
conferences. They tell a powerful story. . . Central is operating in a very non-higher education manner. 

They began their transformation work without funding. No one dangled any money. The motivation was 
pure and because of that their ideas have far greater meaning and passion." 

Daniel Seymour, President, Q Systems 
Author and Consultant IBM, AAHE, ACHE 

"Transforming and restructuring universities will not come easily. Without severe external pressures to 
reform, few universities will commit and endure. Quality transformation is not for the timid, the 
comfortable, the fastidious, or the selfish. It is the preserve of people of strong beliefs and actions. Not 
the talkers but the doers, like Hewlett Packard and Hallmark Cards. Central Missouri State University 
may be just such a needed benchmark and role model for reform in higher education. . . Central has made 
a fundamental commitment to restructure the whole university focused on a sound new core learning 
process. The CPI model can provide leaders with an effective prototype for reinventing universities, 
connecting them with PK-12, and increasing real productivity and quality as defined by the customer." 

Arthur D. Wainwright, Chairman, Wainwright Industries, Inc. and 
Winner of the 1993 Missouri Quality Award and 
1994 Malcolm Baldridge National Quality Award 

"In developing a ’continuous process improvement’ model for monitoring and guiding the didactic process, 
the University [Central] has taken advantage of the techniques used so successfully in recent years by 
corporate America in achieving dramatic performance improvements. . . It is also my belief that what is 
going on at CMSU will prove in time to be a model of singular importance for other institutions of higher 
learning in our region, as well as across the nation." 

John L. Ait ken, Ph.D, Vice President, Quality Performance, Marion Merrell Dow 

and Director, Excellence in Missouri Foundation 

"As President of The Learning Exchange and a member of Missouri’s Coordinating Board for Higher 
Education, I have opportunities to study and evaluate a broad range of innovation American education. In 
the past twenty years, I have never observed the depth of commitment and leadership toward a new vision 
and system of higher education than that at Central Missouri State University." 

Connie Campbell, Ph.D., President, The Learning Exchange 

"I was impressed with the foresight that you are applying to new ways of handling the education objectives 
and the inclusion of quality." 

Robert W. Galvin, Chairman of the Executive Committee, Motorola Inc. 
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"The serious need for the kind of reform in higher education that Central proposes to effect is apparent. 
The scarcity of response to that need on the part of post-secondary institutions is equally apparent. . . As 
someone who works with numerous institutions, I know of no other one of its size in this country that has 
managed a comparable breadth of effort and perseverance in establishing an educational process based on 
articulated outcomes expected of graduates. Central’s development thus far gives strong indication that 
they are capable of and ready for systemic change as an institution." 

Georgine Loacker, Ph.D, Alvemo College 

"You are doing some really exciting work at Central Missouri and your leadership in assessment and 
quality applications in learning is going to be important nationwide. 

Monica M. Manning, Ph.D, Academic Quality Consortium 

"Yours is the only campus in the country, to my knowledge, that has allowed the idea [TQM] to arise 
from, and simmer among, the faculty first. I predict that your approach will (a) pre-empt diversionary 
problems many other campuses have experienced and (b) greatly increase the likelihood of relatively rapid 
change in the short term." 

Ellen Chaffee, President, Mayville State University 
TQM consultant and author 

"To adequately deal with the crises of quality, productivity, and rising costs in higher education, we need 
pilot sites of successful systemic transformation which succeed not for themselves alone, but also as 
learning and demonstration sites for all other institutions. Central Missouri State is already thinking 
about, and preparing to become, such a demonstration site." 

Michael Brower, President, Michael Brower Associates 

"Most educational leaders don’t have a clue as to what is involved in a Total Quality Program and the 
commitment that is required. . . We will need a few universities with the leadership, commitment, and 
profound knowledge necessary to show that it can be done and how to do it. Central certainly is one of 
the few with capacity to do this." 

Robert E. Harmon, Chairman, Harmon Industries 

"I am profoundly struck by what I found in their [Central] unique educational structure. First, there is a 
dedication to consistency of purpose. Second, a basic understanding and appreciation for profound 
knowledge that they have embodied in their culture. Third, a strong sense of shared values for their 
mission and purpose for continuous process improvement. Last, the most fundamental of all, is their 
desire and awareness for the need to change." 

Dr. Robert E. Bush, Vice President, Northwest Missouri State University 

"During the past five years as an American Council on Education Fellow and as a NCA evaluator, I have 
visited over 60 campuses -- public and private, large and small. Central is the only one which could be a 
model for quality improvement throughout the institution. . . [One] cannot find another institution with 
the leadership, vision and commitment to the monumental ’paradigm shift’." 

Linda L. Lamwers, Assistant Vice President for Academic Affairs 

St. Cloud State University 

"What I found [at Central] was one of the most exciting yet daunting experiments I could imagine for 
higher education. The excitement came from a group of academic professionals who were rethinking the 
purpose of education in modern terms. . . they were asking the fundamental question about how best to 
organize a process to maximize learning." 

Peter C. Bishop, Chair, Studies of the Future, 
University of Houston, Clearlake 



*Note: Quotations excerpted from letters of support for Central’s FIPSE Proposal for Phase V (1994-97) 
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